Thirld Word Blog

Passages I highlighted from The Lessons of History

2020-06-13T00:00:00-07:00

Will and Ariel Durant dedicated their lives to writing a history of the world that spans 11 volumes, 10K pages, and 4M words. Then they summarized the whole thing in the 128-page Lessons of History, which was published in 1968. I'm not sure such a book could be published today, because some of the ideas in it are... well, you'll see. Here are a few passages that I found thought-provoking. As always, don't take these as an endorsement. Read the book and form your own opinions.

On inequality and the concentration of wealth:

Inequality is not only natural and inborn, it grows with the complexity of civilization. Hereditary inequalities breed social and artificial inequalities; every invention or discovery is made or seized by the exceptional individual, and makes the strong stronger, the weak relatively weaker, than before. (chapter 3)

Nature smiles at the union of freedom and equality in our utopias. For freedom and equality are sworn and everlasting enemies, and when one prevails the other dies. Leave men free, and their natural inequalities will multiply almost geometrically, as in England and America in the nineteenth century under laissez-faire. To check the growth of inequality, liberty must be sacrificed, as in Russia after 1917. Even when repressed, inequality grows; only the man who is below the average in economic ability desires equality; those who are conscious of superior ability desire freedom; and in the end superior ability has its way. Utopias of equality are biologically doomed, and the best that the amiable philosopher can hope for is an approximate equality of legal justice and educational opportunity. A society in which all potential abilities are allowed to develop and function will have a survival advantage in the competition of groups. (chapter 3)

On long-term economic cycles:

Every economic system must sooner or later rely upon some form of the profit motive to stir individuals and groups to productivity. Substitutes like slavery, police supervision, or ideological enthusiasm prove too unproductive, too expensive, or too transient. (chapter 8)

Since practical ability differs from person to person, the majority of such abilities, in nearly all societies, is gathered in a minority of men. The concentration of wealth is a natural result of this concentration of ability, and regularly recurs in history. [...] The concentration [of wealth] may reach a point where the strength of number in the many poor rivals the strength of ability in the few rich; then the unstable equilibrium generates a critical situation, which history has diversely met by legislation redistributing wealth or by revolution distributing poverty. (chapter 8)

On governments and revolutions:

Nothing is clearer in history than the adoption by successful rebels of the methods they were accustomed to condemn in the forces they deposed. (chapter 5)

No one man, however brilliant or well-informed, can come in one lifetime to such fullness of understanding as to safely judge and dismiss the customs or institutions of his society, for these are the wisdom of generations after centuries of experiment in the laboratory of history. (chapter 5)

Most governments have been oligarchies -- ruled by a minority, chosen either by birth, as in aristocracies, or by a religious organization, as in theocracies, or by wealth, as in democracies. [...] If the majority of abilities is contained in a minority of men, minority government is as inevitable as the concentration of wealth; the majority can do no more than periodically throw out one minority and set up another. (chapter 10)

Democracy is the most difficult of all forms of government, since it requires the widest spread of intelligence, and we forgot to make ourselves intelligent when we made ourselves sovereign. (chapter 10)

Bikepacking from Pittsburgh to DC

2020-05-04T00:00:00-07:00

Trip dates: 2019 Aug 31 - Sept 8.
Trip report date: 2020 May 4.
Amount of procrastination in between: award winning.

I learned about this bike route while researching the longest rail-trails in the US. Armin had just moved to Pittsburgh, and Sunil had some time off between jobs, so it was the perfect time for an adventure! Credits to them for some of the photos below, especially the better ones.

Overview

If you want to ride this route yourself, here are the main things you need to know. The way from Pittsburgh to DC is about 334 miles (537 km). It starts from 720 ft (219 m) of elevation in Pittsburgh, climbs over the Eastern Continental Divide (2,392 ft, 729 m), and then descends to sea level in DC. The gradient is very gentle, never above 2% or so. But the route is unpaved, so you will go a bit slower than on roads.

The first half of the route follows the Great Allegheny Passage (GAP) trail from Pittsburgh to Cumberland, MD. This trail is about 150 miles, with a surface of crushed limestone that you can ride quite fast on. Here is a trail map and a list of campsites nearby. Most of these campsites are paid, but some of them come with luxuries such as laundry machines and hot showers!

The second half of the route follows the Chesapeake and Ohio Canal (C&O) Towpath from Cumberland to Washington, DC. This trail is about 184 miles, with a surface of hard-packed dirt with lots of roots and rocks. This trail is much bumpier than the GAP, and you will go slower. Here is a map and a list of campsites along the way. There is a free hiker-biker campsite every 10 miles or so, which makes planning easy. These campsites are very basic, with no trash bins and no running water. Most of the time there is a well with a pump, but this water is treated with iodine and has a pretty strong taste. So in practice you will want to replenish your water supplies in nearby towns as much as possible.

While the GAP trail would be doable on skinny road tires, the C&O demands wider tires if you want to be comfortable. I had a 35mm in the front and a 38mm in the back, and I got a single flat on the entire ride, mostly because I didn't listen to my own advice and I started the trip with old tires. Sunil had brand-new 32mm touring tires and he didn't get a single flat. Armin had 28mm road tires and he got an unfortunate number of flats.

Carry insect repellent. I didn't think to bring it, and I got some of the worst bug bites of my life. But it was a small price to pay for the adventure.

As you can see from the Strava Global Heatmap, this route is quite popular with cyclists! There are many towns along the way with bike shops, grocery stores, and restaurants. So you never need to carry large amounts of food, and if something goes wrong, it should be pretty easy to bail yourself out.

The entire route is on a rail trail and towpath, so you never have to worry about cars. Most of the time you're just riding through beautiful forest.

I shipped my bike from California and back using BikeFlights, paying a little over $100 each way. I salvaged a cardboard box from a local bike shop, and managed to stuff all my camping gear into it as well. This was cheap, but disassembling the bike and padding it with foam and bubble-wrap took a long time. Next time I might just fly with my bike in a dedicated case, like Sunil did.

We took six days to ride from Pittsburgh to DC. The plan was to then rent a car and drive back. Armin and Sunil did the responsible thing and stuck to the plan. I was a little less reasonable and decided to ride back to Pittsburgh in the remaining three days, which would still get me back in time for work on Monday. So the itineraries below offer two ways to divide the trip: a reasonable one (6 days, 55 miles / 88 km per day) and an ambitious one (3 days, 110 miles / 177 km per day). Thanks to the abundance of campsites, many other variations are possible.

Day 1: That was easy

(From Pittsburgh to Roundbottom campground.)

Here we are, still clean and fresh, setting out from Pittsburgh.

The trail is paved at first, and a little busy. We cross a few bridges over rivers and railroads.

An hour later, we are out of the metro area and into the forest. The trail is now unpaved, but still very easy to ride. The greenery is luxuriant. We see and hear long freight trains on the other side of the river.

We stop at The Trailside in West Newton and devour some sandwiches for lunch. The place is bustling with cyclists.

Shortly before we reach Roundbottom campground, we detour through Perryopolis to get some groceries for the evening. The supermarket is called Save a Lot, and our total bill is $23. We spoil ourselves with some Creme Betweens, the first of many such Totally Not Oreos that we are to experience on this trip.

At the campsite, Armin cooks a delicious pasta dinner with mushrooms, onions, squash, and sausage for the non-vegan. We use the rind of our watermelon as makeshift bowls.

The campground has a water pump, but the water that comes out of it is brown and only useful for rinsing dishes. Armin takes a dip in the river. The weather forecast promises a thunderstorm for tomorrow. There are a few wooden shelters, but they are already taken. So we set up our tents under the trees and the sky.

Day 2: Things get wet

(From Roundbottom campground to Husky Haven campground.)

The night brings little more than a drizzle. I wrap my non-waterproof panniers in trash bags, and we continue onwards. The scattered rain makes the trail very wet, and by the time we get to Connellsville, we are covered in mud.

Despite the rain, the temperature remains comfortable, and riding under the trees is as beautiful as ever.

We decide to save some time and skip the side trip to Fallingwater, which would involve some additional climbing. Ohiopyle State Park is buzzing with people. We linger on the bridges and watch the kayakers on the river below.

After our lunch stop in Ohiopyle, the trail starts to dry out again. We stop to check out the lake and dam near Confluence, and Armin gets his second flat. Sunil and I rush to the town's bike shop and score some chain lube five minutes before they close.

We pass through the Pinkerton tunnel, the first of our trip so far. Inside it's dark and it smells like my grandma's cellar. Lights would be a good idea next time.

We arrive at Husky Haven, our only paid campsite on this trip. We get what we pay for: hot showers, laundry machines, and a high-pressure water hose that erases all the mud from our bikes (oddly satisfying). The one cafe and the one pizzeria in town are both closed. We resign ourselves to eating Clif Bars for dinner, but then we are rescued from this fate by a gas station convenience store that's still open.

Day 3: Big Savage

(From Husky Haven campground to Potomac Forks campsite.)

At night the thunderstorm arrives in earnest. The sky cracks up with lightning, and heavy rain hammers down on our tents. My cheap single-wall tent starts leaking, and the slow drip-drip-drip of water on my head and feet keeps me awake. In the morning the rain lets up and I finally manage to get some sleep. When I get up, my sleeping bag is completely soaked, and the air pad beneath it is an island in a puddle of water that has accumulated on the base of my tent. In what is probably the luckiest coincidence on this trip, this is our one night at a campground with a dryer machine available.

After about an hour of riding, we cross over the Salisbury Viaduct and admire the wide-open views below. There are wind turbines on some of the surrounding ridges.

In Meyersdale there is a small railroad history museum. They even have a few model trains running. We treat ourselves to some ice cream.

A bit later we reach the Eastern Continental Divide, the highest point on our trip at 2,392 ft (729 m) above sea level. It's all downhill from here.

The next few miles are rich with landmarks. First we pass through the Big Savage tunnel, which is just over one kilometer in length. The temperature drops significantly inside. I am just marveling that such a cool structure was restored and made open to the public, unlike some other tunnels I can think of.

Shortly after that we cross the Mason-Dixon line, which I have to look up on Wikipedia.

From there we are on a sweet and steady descent all the way to Cumberland. Six miles before town, my old front tire finally gives up the ghost. After patching and pumping, there are two alarming bumps visible on the tire, and I give them the duct tape treatment like last time. I get a new tire at the bike shop in town. We also stop for a hearty lunch.

Cumberland marks the end of the GAP trail and the start of the C&O towpath. The downtown area near the trail junction looks stylish and recently built. I am surprised to learn from Wikipedia that it is one of the poorest areas in the US.

The trail is smooth and dry initially, but pretty soon it turns into a bumpy mudfest. I start to understand why people recommend MTB tires for the C&O. Sunil says he is energized by the newly technical terrain.

When we reach our campsite, it's already getting dark. The mosquitoes are vicious. We devour some DIY burritos and debate where to set up our tents. The best spot, next to the one picnic table, is already taken by another cyclist. Sunil and I choose the north side of the trail, where we will be sleeping on a slope. Armin chooses the south side of the trail, where he will be sleeping next to a creepy boarded-up lockhouse.

Day 4: River swim

(From Potomac Forks campsite to North Mountain campsite.)

I hear some noises at night, and I worry about rats finding the garbage bag that we left outside. But when I get out of my tent to check, there's nothing there. Just water dripping from the trees. Everything is shrouded in a mysterious mist in the morning. It feels like we are very far from civilization.

The box of Cremes from last night gets turned into a makeshift fender, which works surprisingly well to keep my rear wheel from spraying mud all over my back.

We ride through the Paw Paw tunnel, which is almost as long as Big Savage. The air inside smells ancient, the trail surface is treacherously uneven, and cold water drips from overhead in a few spots. On the other end, the mouth of the tunnel looks like something out of Tomb Raider.

Later in the day, the trail finally starts to dry out. We stop for a big lunch in Hancock, and top it off with a decadent amount of ice cream.

A few miles before our night stop, we take a detour to see Fort Frederick State Park. There's no one else there. The walls of the fort are 5 meters high (17 feet), but unfortunately there are signs not to climb them. The coolest feature is the shape of the fort: a square with a single entrance and four bastions in the corners, designed to allow shooting at anyone trying to climb over the walls.

Our campsite is right on the Potomac River. We find our way to the water and take a dip. The sun bathes the trees on the other shore in soft light as it sets. The air fills with the sounds of frogs and cicadas. We collect some dead wood and make our first campfire. Our dinner tonight comes in the easy form of freeze-dried meal pouches.

Day 5: Pawpaw

(From North Mountain campsite to Calico Rocks campsite.)

A lot of freight trains pass us by at night. With only 200 feet between us and the railroad, approaching trains sound like they're coming straight at us. In the morning we reluctantly fill our bottles with iodine water from the well. As we start riding, the trail joins the river for a while and gives us some ample views.

We take a detour over some unexpectedly steep hills to see the Antietam National Battlefield. There's an air-conditioned museum there, where I do a very Moldovan thing and bail at the $7 entry fee. Outside in the heat, among the cornfields, there are monuments to the various states that fought in the Civil War. New York's is the most imposing.

We cross the river into West Virginia and eat lunch in Shepherdstown.

Back on the trail, another cyclist gestures at us in alarm and says "please stop." His bike is leaning against a tree, loaded with about four times more stuff than our bikes. We ask him if something is wrong, and instead he says, "Can I offer you some fruit?" He opens a bag full of the bruised, yellowish-green blobs that we've been seeing on the trail lately, and says that these are pawpaw, the state fruit of Ohio. We are hesitant to try them because the guy seems kind of weird and we've never heard of such a fruit before. He asks us where we're from and exchanges a few words in Farsi with Armin. This establishes a bit of trust, and Armin and I eventually try the fruit. Despite their questionable appearance and slippery texture, they taste quite good, a bit like bananas with a hint of citrus.

As we are saying our goodbyes, the guy stops us with another off-the-wall question: "Can I tell you a joke?" "Sure." "A man walks into a psychologist's office wrapped head-to-toe in Saran wrap. The doctor takes one look at him and says, I can clearly see you're nuts!"

Where the C&O meets the Appalachian Trail, there is a cool viewpoint called Weverton Cliffs. Unfortunately the hike there involves a few hundred feet of climbing, and it's already late in the day, so we leave it for another time. The scenery from the trail itself is plenty.

In Brunswick we climb away from the river to the supermarket. The trail takes us through a park, where we cross no fewer than 11 bridges. We score another box of Oreo knockoffs.

At our campsite, the bugs are ferocious as usual. We meet a Texan who says he's been backpacking for four months. He has a giant tent and looks like he's been living there for a while. Our day ends with one more rinse in the river, and one more extravagant dinner prepared at our campstoves under Armin's direction.

Day 6: Mission accomplished

(From Calico Rocks campsite to DC.)

Today we enjoy our last miles of forest before entering the DC metro area. After another explosive flat, we detour through the town of Potomac to get Armin a new tire. We pass a succession of enormous mansions that are, upon further research, still cheaper than a condo in the Bay Area.

Locating the C&O Mile 0 marker proves tricker than expected. We finally find it, with help from Google Maps, after weaving through a crowd of rowers moving their very long boats.

Our final stop is at the National Mall, where we take a final picture to go with our starting one in Pittsburgh. After that we head over to meet Sunil's friend, who has graciously allowed us to crash at his apartment for the night.

Day 7: Speedrun!

(From DC to White Rock campsite.)

On the first day out of Pittsburgh, I was having so much fun that I already fantasized about riding back. Bikepacking solo is a little more intimidating than doing it with friends. But now that I know how flat the trail is and how easy it is to get supplies in the towns along the way, I'm kind of looking forward to being alone among the trees with nothing but my bike and some music. I'm planning to reuse some of the same stops as before, so I don't need to do much additional research. I decide to buy my lunch and dinner each day, so that I don't even have to carry a campstove. If I have to bail before reaching Pittsburgh, I can find a train or a rental car to take me back.

Sunil graciously lends me his USB battery pack and a few extra Clif bars. I'm out before 8 AM. I opt for the paved bike trails out of DC, trading some C&O bumpiness for a bit of unexpected climbing. I find myself pushing too hard early in the day, which I will pay for later. Finally I'm back on the C&O near the Great Falls Park, which we skipped on the way to DC because we went looking for a bike shop near Potomac. I snap a few pictures of the whitewater and hurry on.

To bonk means to hit a wall of fatigue because you've run out of glycogen (although the British apparently think it has something to do with discussing Uganda). I want to avoid this fate, so I gobble a Clif bar at the slightest sign of hunger. Before lunch I've already eaten two. I reach Brunswick on time, just after noon, and reward myself with a big burger. (Sorry, but what did you expect? Bike trip reports are usually about very long miles and very delicious calories.)

Around Antietam I venture inland again, aiming to bypass a particularly sinuous stretch of the Potomac river and save 14 miles. This turns out to be a mistake, as the peaceful shaded river path is replaced by a hot hilly grind that makes my eyes and knees hurt. On this detour through Jesusland I see lots of Trump flags in people's front yards. I stop at a gas station to get a cold drink, and spot a pick-up truck full of bumper stickers such as "A liberal is someone so open-minded that their brains fell out."

Worst of all, Google Maps takes me through a section of Hwy 68 that is closed for road work in my direction of travel. There is no way to get where I'm going without another lengthy, hilly detour, so I take my chances on the wrong side of the road and walk my bike on the non-existent shoulder.

A few miles out of Hancock, I ditch the C&O for the paved West Maryland Rail Trail. Everything hurts at this point, not just the usual aches in my back and bottom, but actual soreness in my leg muscles, which I haven't experienced from biking in a long time. On the other hand, I made it to my intended destination just after 6 PM! This is by far the longest I've ridden in one day.

I walk into a local restaurant and confirm that yes, I want all those toppings on my pizza. I get cold from the AC before it finally arrives. After that, it's just a few minutes to the nearest C&O campsite, where I set up my tent just as it's getting dark.

Day 8: Fixie

(From White Rock campsite to Husky Haven campground.)

The morning is humid and a bit chilly. I scarf down my usual camp breakfast of cheese and crackers, and get back on my bike before 8 AM. It's a relief to find that my legs still work!

About 20 miles into the day, my rear derailleur cable snaps when I try to shift into a lower gear. This serves me right for not replacing it sooner -- I had noticed that shifting was getting harder and harder for a while, but I didn't do anything about it because I was planning to buy another bike anyway... Without any tension in the cable, the chain is on the highest gear in the back, which is way too hard to pedal comfortably on this dirt trail. After much cursing and improvisation, I manage to tie and adjust the cable so that it keeps the chain on a middle gear. I still have my front derailleur, but it gets little use on this flat trail. So I ride the next 40 miles like a fixie.

Cumberland is just as pretty as I remember, with its brick buildings and pointy spires against the forested Appalachians beyond. I entrust my bike to the Cumberland Trail Connection, the same shop where I got a new tire just a few days ago. Meanwhile I go to find lunch, and gulp down three glasses of the most divine iced tea I have ever tasted.

The shifting is so much easier with the new cable. The fix comes just in time, too, for climbing over the Continental Divide. The less-than-2% grade is a lot gentler than my usual climbs in the Santa Cruz mountains, so even with a loaded bike, it's not too bad. The Borden and Big Savage tunnels are just as cool as they were the first time. Exhaustion catches up to me eventually, but the views from the Salisbury Viaduct make up for it.

Back in Rockwood around 7 PM, my only option for food is the gas station convenience store again. But that's OK; their pasta salad and chocolate milk go a long way to satisfy a hungry cyclist. One more long day tomorrow, and I'm done!

Day 9: Home

(From Husky Haven campground to Pittsburgh.)

The morning is cold and foggy. I start riding shortly after 8 AM, and put my gloves on for the first time. For almost half of the day, I feel like I'm flying. At first I suspect a tailwind, or maybe the magical regenerative properties of the chocolate milk I had last night. Later I realize that I'm just on a long downhill. It's amazing what even a 1% slope can do.

This is one of the best days of the entire trip. Riding on an endless rail trail under the trees near Ohiopyle, I enter a kind of trance state that captures everything I love about bike touring. Life seems so clear and simple: just cover some miles, get to the next town, find food and water. Revel in the easy trail, the absence of cars, the rejuvenating green of the forest. It's this hard-to-articulate feeling that I will miss the most in the coming weeks, when I'm back into the fray at work, with all the complexities and ambiguities of the modern world.

Would this magic last if I were to do this all the time? Or does it only feel so amazing because it's rare, a vacation from reality? I wonder about the people who choose to opt out of the rat race, radically cut their expenses, live in a van in a perpetual search for the next great trail. Is their every day like this, or does it also get mundane after a while?

I reach Connelsville around 11:30, still too early for lunch, so I decide to push to West Newton. A bit later, I stumble into a colorful car show in Buena Vista. Soon after that, I'm back in the Pittsburgh metro area. The trail becomes paved, and I have to check the map more often. A few bridges and a few hills later, I'm back at Armin's place. He's out of town, but his cat Zooey seems happy to see me.

Do I recommend this century-a-day touring style? It was a fun challenge, made possible by the unusual flatness of the trail. But it was also hard on my knees, and I'm lucky that I made it all the way back without injuries. If I didn't have to go back to work so soon, I'd probably choose a more fluid schedule, and stop more to hike and explore interesting places along the way. As long as I have to count PTO days though, this is as good as any vacation can get.

2020 Q1 reads

2020-04-05T00:00:00-07:00

I read two really interesting books about economic development in China and other Asian countries:

How Asia Works, by Joe Studwell

Explains the successful developmental strategy that allowed Japan, South Korea, Taiwan, and most recently China to lift themselves out of poverty and become big players in the global economy. Contrasts this with Indonesia, Malaysia, Thailand, and the Philippines, which did not develop nearly as successfully.

Step 1: Perform land reform to give small plots to farmers, and move away from large-scale plantations or collective farms. Owning their land (as opposed to being tenant workers) motivates farmers to produce more food. They apply a high-labor "gardening" approach which leads to massive yield increases. This creates the initial food surplus and cash savings that enable everything below.
Step 2: Establish export-oriented manufacturing firms. This requires a long period of subsidies and protectionism, allowing these firms to catch up to the global technological state of the art. Export discipline means that these firms are forced to compete on the global market, and earn their subsidies only if they show the promise of becoming global leaders (as opposed to second-rate "import substitution" players on a protected home market).
Step 3: Keep central control over the financial system, and deploy its resources to support the goals above. This means capital controls, interest rate controls, and bank-based finance (stock and bond markets are harder to regulate). This enables the long-term investments needed to move a country to higher-value niches of the global economy. A more open financial system would focus on short-term gains instead (real estate, consumer lending) which will not help your country develop.

I was surprised by how much of this goes against Western economic "best practices." Land reform means expropriating large landowners, which violates basic private property rights. Protectionism is considered inefficient, and subsidies for exporters are considered cheating. Controlled interest rates are considered "financial repression" and deregulation of financial markets is considered enlightened. The author's depressing conclusion is that if you're a developing country, the IMF and World Bank are pretty much the worst places you could go for advice.

China's Economy, by Arthur R. Kroeber

Explains the various stages of reform that China went through, and its upcoming challenges. Some points that I found particularly intriguing:

Can China innovate, beyond just catching up to the existing technological frontier? So far, China has not produced many global technology leaders like Toyota / Canon / Nikon / Sony in Japan or Hyundai / Samsung / LG in South Korea. Huawei is the major exception, and you can see how worried the US is about that.
Will China's economy keep growing as fast as it has been? Possibly not, since they are approaching the limits of stage-1 growth (mobilizing labor and resources) and must transition to stage-2 growth (increasing efficiency while the working-age population declines), which is much harder.
Does China "cheat" by stealing IP from other countries? Not any more than the US cheated in the 19th century. All countries steal IP until they reach a developmental stage where the benefits of protecting IP outweigh the costs.
Why is the Chinese middle class not pushing for a more open political system? Among other things, because they are still only ~1/4 of the population. In a more representative system, they would lose out to the interests of the rural population, which is still almost 1/2 of the country.
In the West, political legitimacy is given by winning elections. In China, the government derives its political legitimacy from good governance: running the economy well, and improving people's quality of life. We can expect the communist party to stay in power for as long as they keep running the country competently.

Various articles online

Conrad Bastable writes about Germany's economy. An interesting parallel to the books above. Germany also has an export-driven economy, which would normally make its currency appreciate to the point where it starts to hurt their exports. But because they are in the Eurozone with other less successful countries, this isn't happening.
Gwern writes about improvements in daily life since the 1990s. Remember floppy disks and dial-up? It's hard to notice when things get gradually better, like the boiling frog but in the opposite direction. It's good to know that someone's been paying attention.
Financial Samurai writes about how children's unequal affection towards their parents feels from the point of view of the parents. I've never thought about this before.
Pedestrian Observations: Why American Costs are So High when it comes to building infrastructure.
Slate Star Codex: I Can Tolerate Anything Except the Outgroup. Tolerance is hard and if you think that you're criticizing your own tribe, you're probably wrong.
Slate Star Codex: Staying Classy. Offers some ways to think about the very taboo topic of social classes in the US.
The Atlantic: The More Gender Equality, the Fewer Women in STEM.
The Atlantic: The Blog Comment That Achieved an Internet Miracle.
Choice quote (originally from Scott Aaronson):

"There are no task forces devoted to it, no campus rallies in support of the sufferers, no therapists or activists to tell you that you're not alone. There are only therapists and activists to deliver the opposite message: that you are alone and it is your privileged, entitled, male fault."

Fiction

Finally, a friend recommended the short story "Liking What You See", by Ted Chiang. In it, people develop a procedure to block their perception of beauty, so that they can judge one another purely based on other merits. What could possibly go wrong?

2019 Q4 reads

2019-12-25T00:00:00-08:00

Fun stuff:

I found my new favorite word: cattywampus.
I also found an awesome podcast full of dark humor: Welcome to Night Vale.
Scott Alexander writes about enlightenment.
Anton Dubrau writes about tunnel boring machines.
Casey Handmer writes about the SpaceX Starship and Starlink, and what cities on Mars will look like.
Some folks are trying to build a car-free neighborhood in Arizona.
Some fun recent UFO stories.

Serious stuff:

Considerations on cost disease (and comments) from Slate Star Codex discusses why things like healthcare, education, and infrastructure seem to be getting more expensive without getting any better. See also Patrick Collison's examples of how quickly we used to be able to accomplish big projects. And some ideas about why infrastructure is more expensive to build in the US than in other countries.
An essay by Rob Riemen that describes how fascism emerges in a democratic society with a moral / educational vacuum. "Fascist techniques are identical everywhere: the presence of a charismatic leader; the use of populism to mobilize the masses; the designation of the base group as victims; and the direction of all resentment toward an 'enemy.'" It seems like the two US parties choose different scapegoats (immigrants? billionaires?), but the mechanism of exploiting resentment is the same.
Related: an essay that argues that the real "class war" is between the capital-gains class and the professional elite (not the working class).
Related: an SSC piece about billionaire philanthropy. Some people think the money would be better spent if the government taxed it away... They must be far more optimistic than me regarding the competence and efficiency of the federal bureaucracy.
Some insights into the workings of the management consulting industry. (The author does seem a bit one-sided; I'd love to see a rebuttal.)
A long read about the nuclear accident at Chernobyl. Lots of details that I didn't know before, like the fact that personal dosimeters were made illegal in Kiev, and that all the children were evacuated from the city at one point.

2019 Q2 and Q3 reads

2019-10-14T00:00:00-07:00

A Planet Money episode on planned obsolescence.
Idle Words on securing congressional campaigns.
A self-contained ecosystem in a bottle that is still thriving after it was sealed from the outside world in 1972.
"Unauthorized art" being sadly removed from the Albany Bulb, a place I visited a few times when I lived in Berkeley.
A friend is running for office on a pro-housing platform!
Wait But Why has a grand new series unfolding!
An inspiring Youtube series about a couple biking across America. I was sad to learn that they broke up later.
An insightful exploration of why the awesomeness of bike camping is so difficult to convey to others.

Dismantling Your Own Dreams

2019-07-10T00:00:00-07:00

When you join the CBT club, the first thing they teach you is that you're wrong about everything. It isn't that everybody hates you and you'll be alone forever because of that embarrassing snafu at the holiday party. It's that you're over-generalizing from a single incident, you're mind-reading other people's thoughts without evidence, and you just haven't learned how to be kind to yourself yet.

They teach you that how you feel is a result of how you think, and not the other way around. They teach you to trust your thoughts less. They teach you to notice the self-critical voice inside your head and question it; find all the ways in which it's wrong. A favorite CBT exercise is the two-column technique, where on the left side of the page you write down what your inner critic says, and then on the right side you write a more self-compassionate rebuttal, like what you would say to a friend if they felt that way. Left side: I suck at dating. Right side: I've had some good dates, and I deserve credit for trying. Left side: The boss hated my presentation. Right side: I don't know that, since I haven't heard any feedback one way or another. I'll just ask about it tomorrow.

It's like a perpetual game of Gollum vs Smeagol, where the goal is to understand that Gollum does not have a monopoly on the truth, and in fact his views are often unjustifiably bleak. Then you gradually develop Smeagol's voice as a kinder and more realistic replacement, until it becomes effortless and automatic, and Gollum gives up in disgust and away he goes, precious.

This works well for statements that are easily falsifiable, and where Smeagol can confidently say that his statement is closer to the truth. Gollum: I am a failure. Smeagol: Clearly I don't fail at everything; I successfully brushed my teeth this morning. (Someone please draw me a picture of Smeagol brushing his teeth; I will love you forever.) Gollum: I'll never find another creature to make little Gollums with. Smeagol: I may find such a creature -- nothing in the future has probability 0% or 100%, so I can't say that I never will.

This more-true-than view of CBT is hard to argue against, at least if you see truth as a self-evident good thing. But it hits a limit with statements whose truth is more difficult to determine. Consider the pair of statements: I will [probably / probably not] find a suitable life partner. Figuring out which one of these is more true requires picking a frame of reference. If my frame of reference is my own life experience so far and nothing else, then the answer is no -- I haven't found someone yet. If my frame of reference is all people alive today, then the answer is yes -- most people eventually find someone. In between these extremes are a myriad reference points that try to incorporate some prior information about my personality and preferences, then look at statistics about other people who fall into the same bucket (MBTI type, age at first relationship, breadth of social network, etc). Which of these reference points is more true?

When the more-true-than tower falls, CBT retreats one level below, into the more-constructive-than fortress. Even when I can't determine which statement is more true, I can see that believing one or the other will lead to very different behavior on my part. And some of those behaviors might take me closer or farther away from the outcomes that I want. In this view of CBT, I choose what I believe not based on what is more true (undecidable), but based on what is more self-compassionate and more optimistic and more likely to get me what I want. This is clearly a less comfortable rule of thumb for choosing my beliefs than the truth-based approach -- see road to hell, good intentions, etc. But it still makes sense if I optimize for my own happiness, and the only person responsible for my happiness is me.

Constructing counter-arguments in your head is a powerful piece of mental software, and like all tools, it can be used for good and for evil. Underneath the more-constructive-than fortress lies yet another layer, the rationalize-anything basement. Here the truth heuristic doesn't help, because you're considering possibilities / future paths / different branches of the fig tree, neither more true or false than the others. The constructiveness heuristic also fails, because you're not even sure of what you want. Or worse, in the true CBT spirit of questioning everything, you doubt the validity of what you think you want. Maybe you want to be a rock star but hate practicing the guitar. Maybe you want to be a writer but you don't enjoy the process of writing. Maybe you're in limerence with the idea of someone but not the person herself.

And so you start dismantling your own dreams, scrutinizing every piece, looking for counter-arguments using exactly the same techniques that they taught you for fighting Gollum. Left side: I dream of my own house in Portland with deep greenery in every window and raspberries growing in the backyard. Right side: Yeah but renting is more financially efficient. My friends are here and I don't know anyone in Portland. (Cheryl Strayed doesn't count.) The jobs here pay much better. There's good biking weather for ten months out of every year. And I can buy raspberries at the farmers' market. Maybe I should just decorate my room a little and stop daydreaming about greener grass.

In this way you can convince yourself to be content with your current situation. You can talk yourself out of any "dreams," because why give up something good and familiar in favor of a big unknown? Something about this seems wrong to me. How do you distinguish between dreams that are worth throwing everything away for, and dreams that are better left as fantasies? I'm still looking for a good heuristic.

2019 Q1 reads

2019-04-27T00:00:00-07:00

Books:

Bad Blood, a book about the Theranos debacle. The most interesting (and unsettling) part was about the aggressive lawyers / private investigators that they used to intimidate and coerce whistle-blowers. If you don't have money to legally defend yourself, you are screwed.
To Shake the Sleeping Self, a book about a guy who turns 30, decides to bike from Oregon to Patagonia, and spends the next 16 months doing just that. So inspiring =)
Deep Creek, a book about finding home in the mountains of Colorado. I wanted to read this ever since it was mentioned on Cheryl Strayed's podcast. My favorite quote:

The language of the wilderness is the most beautiful language we have and it is our job to sing it, until and even after it is gone, no matter how much it hurts. If we don't, we are left with only a hollow chuckle, and our big brains who made this mess, our big brains that stopped believing a long time ago in beauty, in everything, in anything.
In Patagonia, in which I found a gem! In chapter 72, the author has this quote from Charley Milward's unpublished sea-stories (see "some sources" at the end of the book):

... waking one night in a norther off Valparaíso, the ship on her beam ends and his friend saying: 'Go to sleep, Ugly, you little fool, and you won’t feel the drowning', and then thirty-six hours on the pumps and the cheers of the men as the pumps sucked dry.

I of course immediately connected this to The Island by the Decemberists, which has the following lines:

Go to sleep now, little ugly
Go to sleep now, you little fool
Forty winking in the belfry
You'll not feel the drowning
You'll not feel the drowning

The best part is: most online attempts to explain this song focus on the references to Shakespeare's The Tempest, while completely missing the In Patagonia reference, even though the word "Patagon" also appears in the song. (Some get it right.) I love discovering hidden connections like this!

Cool stuff online:

Why recycling may not be as effective as we'd like to believe.
An inspirational article from a couple who travels by bike:

My Greater Fear is that I will rot beneath a matrix of fluorescent lights staring at the carpeted walls of a cubicle, or that I will wake knowing exactly what I will be doing every minute of every day for the rest of my waking life, or that I will wait until I am old and enfeebled to give myself permission to live.
Another inspirational article from a world traveler's blog. We don't respect the shortness of life enough. 50 more blueberry seasons? Let's waste this one spinning the corporate hamster wheel, wishing we were richer and more attractive, while our dreams just sit there collecting dust.

2018 Q4 reads

2019-01-17T00:00:00-08:00

Books:

The Power by Naomi Alderman: Flips the power balance between genders. A mind trip in the same league as the writings of Ursula K. Le Guin.
How to Change Your Mind by Michael Pollan: The book is pretty positive about psychedelics overall, which makes me a bit suspicious, because Listening to Prozac was also very positive back in 1993, and now they say Prozac is little better than a placebo.
The Way of Men by Jack Donovan: If you liked Fight Club, you will love this. For example:

In the future that globalists and feminists have imagined, [there will only be] more apologizing, more submission, more asking for permission to be men. [...] There can only be more counseling and sensitivity training. [...] There will definitely be more taxes. There will probably be more Byzantine sexual harassment laws and corporate policies and more ways for women and protected identity groups to accuse you of misconduct. There will be more micro-managed living, pettier regulations, heavier fines, and harsher penalties.
The Strange Death of Europe by Douglas Murray: About Muslim immigration into Europe. Written in the wake of the refugee crisis from a few years ago. The book seemed a bit alarmist at times, so I wanted to fact-check some of his stats. The result was a mixed bag:
- p142: "In 2006 the British Medical Association reported that at least 74,000 women in Britain had been subjected to genital mutilation" and p226: "130,000 women in Britain [have] suffered this barbaric treatment." Sadly this seems to check out (source 1, source 2: 66K, source 3: 103K + 24K + 10K). Note the misleading phrasing: Many of these women suffered FGM in their home countries before moving to Britain, where it is illegal.
- p312: "the Vienna Institute of Demography confirmed that by the middle of this century a majority of Austrians under the age of 15 would be Muslims." I found the source; see appendix 2. The claim is true in only two "migration = Muslim" scenarios, which assume "the share of Muslims among migrants increases linearly up to 100% in the period 2026-31". This is a pretty extreme assumption used as an illustration, not a prediction AFAICT. ("This scenario illustrates the relative influence of immigration of a specific group on the overall religious composition of the country.")

Podcasts:

Tweak the Vote from Radiolab (about ranked-choice voting)
In the No, also from Radiolab (consent is stranger than you think)
27: The Most Perfect Album from More Perfect (listen for yourself)
The Gun Show, also from More Perfect (the history of the second amendment is stranger than you think)

Cool stuff online:

The Coddling of the American Mind (on why trigger warnings may be hurting more than they help; also: learn some CBT, people!)
Sort by Controversial from Slate Star Codex (I wish I could write like this)
The Parable of the Talents, also from Slate Star Codex (on forcing yourself to do things, vs. discovering the things you're good at that take zero willpower)
Archipelago and Atomic Communitarianism, also from Slate Star Codex (on exit rights, freedom of association, and allowing society to fracture into subcultures)

So Close to 65,536

2019-01-12T00:00:00-08:00

Let's just say I had a very long flight...

Sorrowful Evening ("Seară Tristă" by George Bacovia)

2018-12-09T00:00:00-08:00

Today has been a sad winter Sunday, so here's a poem by George Bacovia. First, the original version in Romanian:

Seară Tristă

Barbar, cânta femeia-aceea,
Târziu, în cafeneaua goală,
Barbar cânta, dar plin de jale, --
Și-n jur era așa răscoală...
Și-n zgomot monstru de țimbale
Barbar, cânta femeia-aceea.

Barbar, cânta femeia-aceea...
Și noi eram o ceată tristă --
Prin fumul de țigări, ca-n nouri,
Gândeam la lumi ce nu există...
Și-n lungi, satanice ecouri,
Barbar, cânta femeia-aceea.

Barbar, cânta femeia-aceea,
Și-n jur era așa răscoală...
Și nici nu ne-am mai dus acasă,
Și-am plâns cu frunțile pe masă,
Iar peste noi, în sala goală, --
Barbar, cânta femeia-aceea...

I think the best way to experience this poem is via its song version by Maria Mocanu, a Moldovan singer with an appropriately sonorous voice.

I found two translations of the poem into English: here and here (p24). I was a bit disappointed with them, because they didn't try very hard to preserve the rhyme structure (ABCBCA x2, ABCCBA) or the meter (9 syllables in every line, stressed x/x/x/x/x). So here is my own attempt at translation, where I took some liberties with the details, but kept the rhyme and meter mostly intact:

Sorrowful Evening

A savage song she sang, that woman;
The bar was empty when night found us;
A savage song, but full of hunger
And the upheaval all around us
And the piano's monstrous thunder;
A savage song she sang, that woman.

A savage song she sang, that woman.
Our sorry bunch shifted and drifted
Through clouds of smoke, unhappy fellows;
The worlds we dreamt -- never existed...
Intoning long, satanic echoes,
A savage song she sang, that woman.

A savage song she sang, that woman,
And the upheaval all around us...
We never even ventured homeward;
We cried with foreheads on the counter.
The empty bar, and from above us,
A savage song she sang, that woman.

Grand Teton and Yellowstone

2018-11-12T00:00:00-08:00

Trip dates: 2018 Aug 10 - 19. See all Grand Teton and Yellowstone photos.

Besides the Oregon Coast bike tour, this was my other big trip of 2018. This time I didn't manage to recruit any friends with enough vacation days, so I went by myself. On a Friday afternoon I left work a little early and flew to Salt Lake City, then drove north towards Grand Teton. Interstate 15 had a speed limit of 80 mph (the highest I've ever seen), and in practice people were going at 100. The "eco" light on my rental car turned off around 95.

The road got increasingly empty as I drove farther into the night. There was a one-way section with a long red light, where I got out to pee and realized how alone I was. There were no lights other than my car, which I left running. It was just me and the Milky Way.

Around 2 AM I pulled onto a forest road near Alpine, WY, and settled in for the night. After a few hours of fitful sleep in the back seat of the car, with recurring dreams of waking up to a $75 ticket, I got up and moved on. (No ticket! Whew.) I arrived at the Signal Mountain campground around 9 AM, and grabbed one of the two campsites that weren't claimed yet.

Grand Teton Day 1: Carry Bear Spray

The first hike of the day only climbed a few hundred feet, but my heart beat like I was running a marathon. It would take me a while to acclimate to the elevation. The mountains were covered in a disappointing haze from some remote forest fires. For a while I wished that I had tried harder to find some friends to join me, but that feeling went away in the coming days.

At every trailhead there was a scary sign about bears. "Be alert. Make noise. Carry bear spray. Avoid hiking alone. Do not run." I stepped into a convenience store planning to get some bear spray, but balked at the $50 price, especially since I couldn't take it on the plane home if I didn't end up using it. So I made the questionable decision to continue on my trip without bear spray.

In the hour before sunset I drove up to the top of Signal Mountain. On one side, the landscape looked like something out of a Warcraft II-era game. On the other side, the sun was slowly descending over the Tetons. I sat down with a giant watermelon and scooped it up with a spoon, deflecting envious glances from other sunset watchers.

Grand Teton Day 2: Of Elk and Moose and Child Carriers

The plan for today was to hike Granite Canyon from the top of Rendezvous Mountain. This trail was one long 4,000-foot descent, with the Jackson Hole aerial tram doing the hard work of getting you up at the beginning. The engineering of the tram was pretty impressive, but the views were not (too much smoke).

The first part of the hike was a feast of wildflowers -- spring comes late in the mountains. The second part was along a picturesque river -- I dipped my toes in but it was obscenely cold. On the third and final part, some animal lept a few feet away and scared the crap out of me -- it turned out to be just an elk.

Later I swung by Phelps Lake just in time for sunset. There I saw a French-speaking couple with a young kid walking by himself, and another one in a baby-carrier backpack. I found this really inspiring; maybe having kids and exploring national parks are more compatible than I previously assumed.

Driving back at dusk, I saw a moose staring ominously from the side of the road.

Grand Teton Day 3: The Moods of Mormon Row

This was a prolific day in terms of photos. I forced myself to get up at 6, and drove to Schwabacher Landing where I heard there was wildlife to be seen. But the only things I saw were a paddling of ducks and a horde of DSLR-wielding tourists. I joined the latter for a while.

My next stop was Mormon Row, where the abandoned homesteads of early settlers still stood. It seemed incomprehensible that people would abandon or sell such prime real estate, but I guess starvation due to failing crops can be a pretty powerful motivator.

Then the weather changed, and the place looked like a horror movie set.

I spent the rest of the day rambling along the edge of Leigh Lake all the way to Bearpaw Lake. It was supposed to be a flat and easy 8 miles, but my feet were hurting after yesterday. I made it back to my campsite pretty early, and ate dinner without a headlamp for the first time.

Grand Teton Day 4: The Mother of All Day Hikes

I had a 19-mile loop hike planned for today, but after how my feet felt yesterday, I thought I would turn around after 7 miles. When I got to Lake Solitude though, I still felt pretty fresh, and the scenery was so breathtaking that I decided to keep going.

Paintbrush Divide was rocky and barren, and so high up that it still had some snow in the middle of August. On the way down I was greeted by a family of mountain goats! (Mom, dad, and two kids.) They were as big as cows, and amazingly adept at descending on steep rocky and snowy slopes.

Once I got down to a lower elevation and my brain started working properly again, I realized that my decision to push forward with the 19-mile loop had two serious flaws. First, I didn't bring enough water. Second, I had to catch the last boat across Jenny Lake at 7pm, or risk being stranded with the bears.

I took to jogging on the flatter sections of the trail, and my feet felt like tenderized meat. The path to Lake Solitude had been quite crowded, despite the name. This current path was nearly deserted though, and I was a bit worried about animals. I clapped my hands loudly every few minutes, trying to make as much noise as possible.

In the end I made it to the boat on time, and rewarded myself with a sports beverage from the convenience store on the other side. This was a bad idea; the drink was basically all sugar, and gulping a liter of cold water after hiking for eight hours shocked my temperature regulation system so much that I started shivering and had to sit down.

Despite all the poor decisions, this was a pretty epic day hike (my longest ever!), and a grand way to end my time in Grand Teton. I felt grateful for the long summer days, and also for my apparently invincible legs. On the road back to my campsite, this place gave me one final, unexpected gift: a horse paddock with sprinklers dancing against the setting sun.

Yellowstone Day 1: Too much, too bright, too powerful

In the morning I paid for a hot shower at Signal Mountain campground, and it was the best $10 I ever spent. Then I drove to Canyon campground in Yellowstone, which was much farther than I realized. The park itself was huge and kind of intimidating; I looked at the map and didn't know where to start. Finally I picked Norris Geyser Basin since it was relatively close.

This place was so alien and incredible. Blue pools opened into the earth like some kind of portals from another world. Fumaroles exhaled sulphur from their endless lungs. Hot water snaked in rivulets under the boardwalks, with thermophiles tracing its path in unholy streaks of orange and teal. In some places the water bubbled with a restless energy, or exploded upwards in angry geysers. Near the hottest features the trees were dead and limbless. Only their blackened trunks remained, petrified, reaching pointlessly skyward like soldiers who stood too close to the blast.

This was where the human realm ended, and something altogether different (and possibly Lovecraftian) began. I felt a strong impulse to defend and protect these irreplaceable sights, and awe at the foresight of those who made this decision 150 years ago. To live to see this place was the luckiest I've ever felt. The poignancy of this caught me off guard; it was a kind of spiritual experience that even this stone-cold atheist could appreciate. It reminded me of that one time on the California coast when I felt similarly cracked open.

After much staring and wandering, I drove north to Mammoth Hot Springs. Here was another place where water behaved in strange and unexpected ways.

I always thought that moving water worked against flatness, by slowly carving a riverbed wherever it flowed. But here the water was so full of minerals that it worked towards flatness, by plugging any places where a tiny riverbed might start to form. This shaped the landscape into travertine terraces, staircases of water that ranged in scale from a few millimeters to about a foot high.

After greedily traversing the entire graph of boardwalks built up in the area, I got some groceries from Mammoth and started driving back to my campground. The views on Grand Loop Road were spectacular, especially in the golden hour. This is what the American West must have looked like before it was "civilized".

Around dusk I got into a traffic jam caused by a giant freaking buffalo heedlessly walking in front of a long line of cars. I filmed it as it passed right by my open window, within arm's reach, and definitely closer than it would normally be safe to approach. Then I sped off and couldn't stop laughing. Only in Yellowstone...

Yellowstone Day 2: I Can Feel Their Blue Hands Touching Me

I got up early because of the cold. Even in late summer, the nights approached freezing temperature at this elevation. I drove to the Mud Volcano area, arriving to an overcast sky and muted colors. The place looked foreboding, with thick columns of steam hissing out of the earth, like smoke from a burnt village.

By the time I got to the West Thumb Geyser Basin, the sun was winning its battle against the night, and the world was bathed in a perfect morning light. Set right on the edge of the vast cold expanse of Yellowstone Lake, this place was especially enchanting. I spent an hour or two staring into the turquoise pools of the hot springs, imagining what kind of magic Murakami might conjure out of them had he been there. I'd never seen natural features so improbably beautiful.

Eventually I got hungry and moved on. Driving towards Old Faithful, I was surprised to find myself on a 2+2-lane highway, complete with a raised, clover-leaf-type exit. This seemed out of place for a National Park, and I wondered if it had been built in an earlier time, when people saw all the promise of the emerging automobile, and none of the consequences of having a country full of them. (Yellowstone is the oldest National Park, established in 1872. The Model T didn't arrive until 1908.)

Having just missed the Old Faithful eruption, I walked into the visitor center and learned that they could predict the next one within +/- 10 minutes. Just enough time to climb up to an overlook and watch the spectacle unfold. After that I found my way to Solitary Geyser, which erupted every 5-7 minutes, and I stayed to see a few iterations (the second was weak; the third made up for it). I caught Castle erupting from a distance, although at this point, I had to ration space on my camera's SD card, and I was feeling a bit desensitized to all this magic.

Yellowstone Day 3: Where I End and You Begin

Artists' Paintpots looked like a haunted forest in the morning. The smell of sulphur only strengthened the impression that something evil was going on. I made my footsteps extra quiet around the gurgling mudpots, lest they awaken Cthulu.

After two days of shuffling along boardwalks, I was ready to go on a proper hike and leave the other tourists behind. Unfortunately I was not the only one with that idea; the trail to Fairy Falls was a superhighway. The waterfall was impressive enough, but the real reward was Imperial Geyser, about a mile farther out. Here I finally had the place to myself. And what a place it was, when the forest shed its ordinary facade to reveal a steaming turquoise pool and an exuberant gusher.

When I got to the viewpoint above Grand Prismatic Spring, the sky was mostly cloudy, and the spring's famous colors were subdued. What caught my eye was the area next to it, where the plants must have boiled alive, and the travertine stretched across the earth like so many scars. In this landscape of devastation two blackened trees remained standing, their reflections and shadows like the hands of clocks that continued running long after there was anyone left to read the time.

While those clouds deigned not to ruin my day, they showed no such mercy for my campsite. Everything was cold and dripping when I arrived. My cheap tent fought a brave battle against the rain, and lost. The sleeping bag inside was completely soaked. With nothing dry to sit on, I ate some dinner standing up, then put on all the layers I had, and braced myself for another night in the back seat of my car.

Yellowstone Day 4: I Have It All Here in Red Blue Green

Somehow I survived without my sleeping bag, and did not have to drive out of the park to a lower elevation, like I had feared. In the morning I took a long detour through Lamar Valley. There were herds of bison, as promised, and I wished I had a long-zoom lens.

Then I returned to see the Grand Canyon of the Yellowstone. The weather was getting dicey again; I hiked to Point Sublime and then ran most of the way back, thinking that in case of lightning, the exposed lip of the canyon was the worst possible place to be. Did I hear some static crackling? Maybe I just imagined it.

On the north side of the canyon there were two epic, deafening waterfalls, but before I got to explore the trails there properly, it started raining for real. I suppose I got pretty lucky with the weather on this trip; today was meant to show me how much worse it could've been.

I drove back to the Old Faithful area, which cloudy but still relatively dry. Greedily I walked the remaining paths, knowing that this was my last day in Yellowstone. A drizzle came on and off, and the crowds thinned. As the daylight waned, I was the only one left roaming among the mysterious blue pools, the sulphur-spewing gashes, the rusty wounds in the earth.

I was lucky enough to catch Daisy erupting, and the violence of it at dusk gave me a thrill of primal fear. It made a haunting sound when it subsided, like a train departing: click-clack, click-clack, click-clack. When it got so dark that my camera couldn't focus, I knew that it was time to go home.

Closing Thoughts

After one final night at the campground, I packed up my things, took a glorious shower, and settled in for the long drive back to Salt Lake City, where I had a few hours to walk around. Like San Francisco, the place was swarming with hobos. Unlike San Francisco, there were some official "don't support panhandling" signs. I was impressed with the architecture of the temple and the surrounding area (all private property), and happy to see some equally impressive secular buildings: the library, city hall, the state capitol, some old mansions.

At the airport I accidentally tested the recall of the x-ray detection system. I forgot my foldable camping knife in my carry-on, and they found it. I had to go back and put it in my checked-in luggage, which they luckily allowed me to do. Also, I almost flew home with the spare key for my rental car, which I also went back and returned. This is why I always get to the airport with time to spare...

One thing I didn't mention earlier is that I saw a lot of cool RVs on this trip. Especially intriguing were the minivan-sized RVs from a New Zealand company called Jucy. I have a weird fascination with younger people who choose the traveling lifestyle for a longer period of time, and I wish I had made an effort to start some conversations and learn their stories. Maybe next time.

As usual I measured my mustache by tallying up trip expenses:

Flights SFO-SLC plus baggage fees: $286
Car rental: $340
Gas: $111
Campsites: $264
National Parks pass: $80
Misc (Lyft to airport, aerial tram in Jackson, etc.): $90
Groceries and food (rough estimate): $200

This amounts to $1,371 for a ten-day trip. The most obvious opportunities for optimization were the car, gas, and campsites -- these could've easily been split in four had I convinced some friends to come with me. But while this trip wasn't cheap, I had the time of my life, and that's what money is for after all.

Oregon Coast Bike Route

2018-10-19T00:00:00-07:00

Trip dates: 2018 Aug 31 - Sept 9. See all photos here.

About a year ago, I saw the Oregon Coast Bike Route mentioned in a travel book, and I just knew I had to do it. Voitek joined, and we had a gang! With a little luck and a lot of duct tape, we cycled from Astoria all the way to Crescent City, traversing 458 miles with 20,000 feet of elevation gain over the course of a week. That's 737 km and 6100 meters for non-US readers.

(Note: The above numbers are based on Google Maps. The Strava distances below are over-estimates, because I didn't bother to pause the recording when we got off our bikes to explore or eat. You can see the more accurate numbers from Google Maps below each Strava widget.)

Day 0: Border Crossings

(From Astoria to Fort Stevens State Park. Data from Google Maps: 16.4 mi / 26.4 km with an elevation gain of 170 ft / 52 m.)

After twenty hours on a train and another three hours on a bus, we made it to our starting point of Astoria. We chose to travel this way because it didn't require us to disassemble our bikes. From there we rode over the bridge to Washington, just to claim the "Entering Oregon" sign.

Then we turned right back and biked to our campsite. It was already past 10 PM. For $8 per person per night, the hiker-biker campsites in Oregon state parks were pretty amazing. The bathroom taps had hot water, there were free hot showers, and they even had a fancy bike repair station and some lockers with USB charging slots, donated by REI.

Day 1: Duct Tape

(From Fort Stevens State Park to Nehalem Bay State Park. Data from Google Maps: 55.0 mi / 88.5 km with an elevation gain of 3,200 ft / 975 m.)

In the morning we explored Fort Stevens, where soft moss hung from the trees, a hundred-year-old shipwreck rusted on the beach, and fishermen staked out the mouth of the Columbia river.

For lunch we stopped in Seaside, where we eyed the vacation homes built right on the beach. After that we took a detour to Ecola State Park, which involved some punishing climbs. At Indian Beach, Voitek went for a swim while I went for a short hike. Both endeavors ended in disappointment. I got stuck in mud up to my ankles, and didn't see any views because of the thick fog. Voitek learned that the water here was much colder than his CA wetsuit was designed for.

To regain some warmth, we got hot chocolate in Cannon Beach. Here Haystack Rock stuck out of the water like the head of a giant whale. Conscious of the few remaining daylight hours, we hurried south.

We wanted to catch the sunset from Oswald West State Park, but on the way there I got a flat. After quickly patching the tube, I noticed that the tire itself had a noticeable bump in that spot. This was very bad news, as it meant I would get more flats at best, and an exploded tire at worst. The next bike shop was 100 miles south. We decided to wrap the wounded section with duct tape and hope for the best. Voitek couldn't stop laughing at the crudeness of the fix.

We missed the sunset this way, but at least we ate a delicious Thai dinner in Manzanita. After getting some breakfast snacks and more duct tape, we biked to our campground for the night.

Day 2: Apples & Cheese

(From Nehalem Bay State Park to Cape Lookout State Park. Data from Google Maps: 49.3 mi / 79.3 km with an elevation gain of 2,200 ft / 673 m.)

The Miami Foley road was a gem. It led us through peaceful Oregon countryside, lush with greenery and away from the noise of the highway. We stopped to eat some apples from a tree on the side of the road. They tasted just like home. Later we saw some "giant mozzarella balls" in the fields, and wondered what they were. It turns out that even hay gets wrapped in plastic these days.

We stopped at the Tillamook Creamery for some free samples. The place was swarming with tourists. Heading west along the Three Capes loop, the wind almost knocked us off our bikes. We approached Cape Meares via highway 131, which proved formidably hilly. (Later we learned from other cyclists that Bayocean Road was quite passable, despite the "road closed" signs.)

At Cape Meares we walked the short trail to an 800-year-old Sitka Spruce, the largest in Oregon. The earth around the tree was strangely pliable under our feet, as if hollow under a layer of roots. The Cape itself offered panoramic views of the coast, and a lighthouse with a Fresnel lens that we could see up close.

Biking south again, I got a flat in my front tire. Thankfully no duct tape was necessary this time. We stepped into a small grocery in Netarts around dusk. The shopkeeper was from Brooklyn, and we wondered what nightmares (real or imagined) brought him to a town of 744 on the other end of the continent. The only food place still open was a bar with elk skulls on the walls. As was becoming our habit, we rode the remaining six miles to the campground in the dark. The hot showers were glorious.

Day 3: A Mountain of Sand

(From Cape Lookout State Park to Beverly Beach State Park. Data from Google Maps: 61.4 mi / 98.8 km with an elevation gain of 3,040 ft / 926 m.)

Cape Lookout was blanketed in fog in the morning, so we skipped that hike. Near Sandlake we saw people riding dirt bikes on sand. We asked about renting some, but they only had ATVs. In Sitka Sedge we hiked along a marsh, through coastal forest, and onto a beautiful but desolate beach.

At Cape Kiwanda, the orange cliffs and white sand created a memorable landscape against the distant forest and the deep blue water. Strange stone formations offered gentle slopes on one side, and deadly drops on the other. Waves crashed against them, sending spray many meters into the air.

After a slow climb to the top, we ran down the surreal mountain of sand that formed the eastern end of the cape. We wished we had sleds. On the beach, a guy was neck-deep in sand, digging a hole with a shovel. Voitek asked him why, and he said he just started digging every morning, to see how far he would get. I suppose this is not that different from traversing 400 miles on a bicycle. I find such a trip immensely appealing, but I've struggled to explain this to some coworkers. Why not just drive?

By the time we were done with lunch (a disappointing faux-Asian place in Pacific City), it was almost 4 PM. We took Old Highway 101, a beautiful and nearly empty road through dense forest, which sometimes felt like a tunnel under the trees. After a long climb, we earned a sweet downhill that lasted a full ten minutes. In Lincoln City the duct tape finally failed, and I got another flat. I patched the tube and re-bandaged the tire, and we rolled on. By the side of an abandoned house, we saw two llamas that looked lost and confused.

After some great burgers in Depoe Bay, we shivered as we got back out into the cold night air. The road was completely dark, illuminated only by the stars and our bike lights. We missed the ocean views on the remaining seven miles to Beverly Beach, but Voitek did score an awesome long-exposure photo.

Day 4: Rushing Nowhere

(From Beverly Beach State Park to Jessie M. Honeyman State Park. Data from Google Maps: 59.9 mi / 96.4 km with an elevation gain of 2,590 ft / 789 m.)

In Newport I finally got a new tire, Voitek shipped his wetsuit home to save some weight, and we ate an amazing breakfast of omelets and pancakes. This was followed by a few hours of monotonous riding on 101. Voitek used the time to listen to some Spanish podcasts, while I enjoyed that elusive empty-mind feeling that I get on long bike rides. We stopped at Cape Perpetua for some quick photos, but decided we didn't have time to explore.

We rushed to the Sea Lion Caves knowing they closed at 7 PM. Unfortunately, the closed sign was already hanging when we got there at 6. We continued to Florence, rolling our eyes at the name. There were some sand dunes next to a supermarket, so of course we took our shoes off and went for a walk. Later, Voitek found an amazing Thai place for dinner. He then promptly started chatting up one of their female employees, in Thai no less.

Day 5: Dunes!

(From Jessie M. Honeyman State Park to Sunset Bay State Park. Data from Google Maps: 57.2 mi / 92.0 km with an elevation gain of 1,940 ft / 592 m.)

We started our day with a few hours of biking, and stopped for lunch at a sourdough bakery in Winchester Bay). Shortly after that we arrived at the John Dellenback trailhead, our chosen starting point for exploring the Oregon Dunes. We were not disappointed.

The landscape was unlike anything I'd seen before. At the macro level the dunes seemed to stretch for miles and miles, dwarfing the few human figures ambling along the ridges. At the micro level the terrain had a washboard pattern that yielded easily to our feet in some places, and provided a surprising amount of resistance in others. We had some fun bombing down the steepest edges we could find, and observing the sand erase our tracks in a miniature avalanche.

The smooth gradients of the rolling dunes were interrupted here and there by patches of beachgrass and other defiant species. We also saw some undulating tracks that looked like they were made by snakes. A thick layer of forest lay between us and the ocean.

As we walked west, the landscape turned into a marsh, and the trail turned into a boardwalk. The transition from sunny yellows to shady greens was startling. Soon the view opened up into an endless beach. A gang of snowy plovers hung out right next to a sign that said they were endangered.

After the dunes adventure, we biked south to Coos Bay. This was the largest town on our route (pop. 15,967), and the only one that gave us a sketchy vibe. The shoulderless bridge into North Bend was bathed in soft golden light around sunset, but we were too hungry to stop for photos. We ate a big dinner at an Italian place, then pedaled in the dark for another hour to get to our campsite.

Day 6: How Many Devils

(From Sunset Bay State Park to Humbug Mountain State Park. Data from Google Maps: 74.2 mi / 119.4 km with an elevation gain of 3,030 ft / 923 m.)

In the morning we explored Shore Acres State Park, which used to be some rich guy's private botanical garden. We also saw a whale lift its tail above the water, which made me miss my long-zoom camera. We stopped briefly at a sea lion viewpoint and at Cape Arago, but decided to get going since we had a long day ahead.

The way back to the highway was called Seven Devils Road. After the first two hills, it became apparent why. After another two, I stopped counting. At least there was no traffic to contend with. The view at the top was a modest reward: layers upon layers of forested hills, becoming bluer with the distance.

My bike had been sqeaking unhappily for the past few days, so the first thing we got in Bandon was chain lube. We then ate lunch and sampled the local beer. Shortly after that we passed through the dying town of Denmark, which was definitely no Florence.

Our final detour of the day was to Cape Blanco. The rocks along the shore made for some striking scenery, and we lingered to watch the sunset. On the trail back, we almost stepped on a porcupine.

We stopped for dinner at a bar in Port Orford. The building was a brand new construction in the log cabin style, surprisingly grandiose for a town of 1,133. The band had just left, but the burgers were excellent.

Day 7: Coastal Views

(From Humbug Mountain State Park to Harris Beach State Park. Google Maps data: 54.6 mi / 87.9 km with an elevation gain of 3,355 ft / 1022 m.)

Our last full day was all about coastal views. They came at us from around every corner, much like on Highway 1 in California.

In Gold Beach we got some restaurant recommendations from a pair of tie-dyed RV-dwelling hippies. We ate a questionable American breakfast, while the restaurant owner took care of some business on the phone, cursing the other side colorfully as soon as she hung up. Later she gave us two pins to stick in the map on the wall. The US and Western Europe were thoroughly represented, but there was only one pin in Romania (hello Cluj-Napoca), and none in Moldova. Time to rectify that.

Later on, we found dessert in a bramble of blackberries growing in a turnout. In another turnout, we saw a guy on a motorcycle gesturing desperately at us. He needed a push, so we gave him one.

In Brookings we tried another one of the hippies' dubious recommendations: a Mexican restaurant in the shape of a converted gas station.

Day 8: Going Home

(From Harris Beach State Park to Crescent City, CA. Google Maps data: 29.6 mi / 47.6 km with an elevation gain of 510 ft / 155 m.)

The trip was over in my mind as soon as we crossed the California border. I felt a bit of pride for having pulled this off, but overall, finishing the trip felt less exciting than being on the road and seeing new things every day. My mind immediately switched to worrying about the long drive home, and about going to work again the next day. Now that I'm back to zero vacation days, all I can do is to plan something even more ambitious for next year!

Final Thoughts

Overall, we were pretty lucky with no rain and no major bike issues. Here are some lessons we've learned from our first week-long bike tour:

Hell yes, we can do it!
State parks with hiker / biker campsites are amazing resources. For a list along the Oregon coast, check out this brochure. For a list in California, use the Find Parks page and select "Hike or Bike Campsites" under "Overnight Facilities".
Padded biking shorts and anti-chafe cream are miraculous. I used to do all my biking without them, and felt sore after about 50 miles. With them, I rode 50-60 miles for 7 days in a row, and felt like I could keep going even longer. I wonder why I never tried these things earlier.
Schedule a rest day. Biking 50-60 miles every day is doable, but having a shorter day in the middle would've made our knees happier.
Plan shorter days, or get disciplined and start earlier in the morning. Most other cyclists were gone by 7-8 AM, while we typically rolled out of camp around 10, and arrived at the next one after dark. I made a naive calculation before the trip: 13h of daylight - 6h of riding (60mi at 10mph) = 7h to rest, eat, and explore. In practice, a lot of this time was consumed by snoozing in the morning, breaking camp, or looking for the next place to eat.
Carry just enough food in case you bonk between stops, but no more. On this trip, we had access to restaurants and groceries every day. I carried too much: six energy bars and a big bag of trail mix, which was only half empty by the last day.
Do a fully-loaded overnight test ride close to home. The weekend before our big trip, we biked to Santa Cruz on Saturday, camped there, and biked back home on Sunday (about 45mi each way). This helped us debug any missing gear, and convince ourselves that our bikes and our legs were up to the task. I was pretty anxious about the big trip before, but after the test ride went well, my confidence went way up.
Get some new tires before the trip, and carry a lot of duct tape, just in case. ;-)

In true Mustachian fashion, here is an approximate breakdown of our expenses on this trip (all costs are per person):

Getting from home to Astoria (Amtrak + bus): $159
Biker campsites ($8 per night per person): $64
Food (with all the riding, we ate 1.5x the normal amount): $230
Bike things bought on the trip (new tire, chain lube, duct tape): $50
Getting from Crescent City back home (one-way car rental + gas): $161

This comes down to $664 per person for an epic week-long adventure. The campsites are the most badass deal. Transportation could be optimized further, for example by having a friend pick us up (one-way car rentals are stupid expensive). The food could also be optimized, but we really enjoyed the luxury of eating out every day, and have no regrets about spending the money.

2018 Q3 reads

2018-09-30T00:00:00-07:00

Books:

Going Somewhere, by Brian Benson, a memoir he wrote about cycling across America with his girlfriend. I resonated with the way he described the pleasure of cycling for its own sake: being in motion, being among trees and lakes, sprinting uphill even when your legs are screaming. The writing is surprisingly honest, for example, he often felt frustrated / impatient with his girlfriend, even though he knew at a logical level that it wasn't fair to expect her to be as fast as him. I wonder if all bike-touring couples go through this (including when the roles are reversed).
Chaos Monkeys, by Antonio Garcia Martinez, a crazy startup story involving players you've heard of, like Facebook and Twitter. Two instructive questions: Does it take a guy as ballsy as him to have a successful exit for a startup? And do acqui-hires typically make $650K/year in stock, or did he just get lucky with Facebook?

Podcasts:

The Border Trilogy by Radiolab, the fascinating and disturbing story of how the US - Mexico border got to be the way it is today.

Cool stuff online:

A timeline of the far future on Wikipedia. This will make you feel small.
The Mastermind, by the Atavist Magazine, a surreal long read about a guy who started some sketchy online pharmacies, and then went much much farther than that. My favorite bit: "We have guys in Somalia buying weapons that are making an army [...] because we were gonna invade an island, Maldives" (so he would have a place to operate from).
Commoditize your complement, an essay that explains why companies make so much stuff available as free / open-source software.
/r/vandwellers and in particular FarOutRide, a couple of Canadian engineers who decided to build a campervan, take a year-long sabbatical, and go do what they love, which is mountain biking. Call me crazy but I find this incredibly inspiring.

2018 Q2 reads

2018-06-15T00:00:00-07:00

Books:

Attached, by Amir Levine and Rachel Heller, a book that challenged some of my beliefs about psychology and relationships. In particular, they argue against the usual "two self-sufficient people" ideal, saying that mutual dependency is normal and desirable. Quote: "If you want to take the road to independence and happiness, find the right person to depend on and travel down it with that person."
Just Ride, by Grant Petersen, a funny book about cycling. He argues against the fixation on bike races, and in favor of a more casual style, which matches my attitude exactly. Although I still think synthetics are better than cotton ;)

Cool stuff online:

The Assasin Puzzle, a cool math brainteaser from the PBS Infinite Series.
The Non-Libertarian FAQ on Slate Star Codex. I found the "competence of government" section particularly thought-provoking.

Linear Algebra Solves a Childhood Mystery

2018-05-17T00:00:00-07:00

When I was growing up in Moldova, we didn't have the Game Boy or other fancy gaming consoles. Instead we had the Brick Game, which cost about $5 and came with Tetris and a few other simple games.

One of the games worked as follows: You start with a 5x5 board on which some cells are "on" and some cells are "off". The goal is to clear the board by setting all cells to "off". You can toggle any cell, but doing so also toggles the four adjacent cells (above, below, left, and right), assuming those cells do not go over the edge of the board.

Here is an example of clearing a board in four moves. The $\blacksquare$ symbol indicates a cell that is "on", and the $\fbox{$\phantom{\blacksquare}$}$ symbol indicates which cell was toggled in each step.

$$ \begin{bmatrix} & & \blacksquare & & \\ & \blacksquare & \blacksquare & & \\ \blacksquare & \blacksquare & & \blacksquare & \\ & & \blacksquare & \fbox{$\blacksquare$} & \blacksquare \\ & & & \blacksquare & \end{bmatrix} \rightarrow \begin{bmatrix} & & \blacksquare & & \\ \fbox{$\phantom{\blacksquare}$} & \blacksquare & \blacksquare & & \\ \blacksquare & \blacksquare & & & \\ & & & & \phantom{\blacksquare} \\ & & & \phantom{\blacksquare} & \phantom{\blacksquare} \end{bmatrix} \rightarrow \begin{bmatrix} \blacksquare & & \blacksquare & & \\ \blacksquare & \fbox{$\phantom{\blacksquare}$} & \blacksquare & & \\ & \blacksquare & & & \\ & & & & \phantom{\blacksquare} \\ & & & \phantom{\blacksquare} & \phantom{\blacksquare} \end{bmatrix} $$

$$ \rightarrow \begin{bmatrix} \blacksquare & \fbox{$\blacksquare$} & \blacksquare & & \\ & \blacksquare & & & \\ & & & & \phantom{\blacksquare} \\ & & & & \phantom{\blacksquare} \\ & & & \phantom{\blacksquare} & \phantom{\blacksquare} \end{bmatrix} \rightarrow \begin{bmatrix} \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} \\ \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} \\ \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} \\ \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} \\ \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} \end{bmatrix} $$

As the game progressed, the boards got more and more difficult, until some board seemed completely impossible to solve, no matter how much time I spent on it. I couldn't figure out, back then, if such a board was genuinely unsolvable, or if I was just too bad of a player to solve it.

Then in 2016, I went through David Poole's textbook on linear algebra to prepare for some interviews. I found the section on finite linear games, which examines a similar game with five lights and five switches. Poole shows that to get from one state of the lights to another, all you have to do is solve a system of linear equations. This made me think of the original 5x5 game I played as a kid, and I wondered what questions I'd be able to answer about it using linear algebra. After much procrastination, this blog post summarizes the results.

Problem setup

Each board can be represented as a vector in the finite field $\mathbb{Z}_2^{25}$, i.e. a 25-dimensional vector where each entry is 0 or 1. So the starting board in the example above can be represented as the following vector:

$$\textbf{b}_1 = [0 0 1 0 0 | 0 1 1 0 0 | 1 1 0 1 0 | 0 0 1 1 1 | 0 0 0 1 0].$$

(I've added some $|$ symbols to make it easier to see the 5 blocks of 5 cells.)

Each action can also be represented as a vector in the same space. So the first action taken in the example above can be represented as the following vector:

$$\textbf{a}_{19} = [0 0 0 0 0 | 0 0 0 0 0 | 0 0 0 1 0 | 0 0 1 1 1 | 0 0 0 1 0].$$

(I'm calling it $\textbf{a}_{19}$ because it is the action of toggling the 19th cell.)

Applying an action to a board means adding the action vector to the board vector. Note that in $\mathbb{Z}_2$, addition is modulo 2, i.e. 1 + 1 = 0. So we get the second board in the example above as $\textbf{b}_2 = \textbf{b}_1 + \textbf{a}_{19}$.

To solve an initial board $\textbf{b}$, we need to find a sequence of actions that, when added to $\textbf{b}$, produce an empty board, for example:

$$\textbf{b} + \textbf{a}_{15} + \textbf{a}_8 + \textbf{a}_{21} + ... = \textbf{0}.$$

Since addition is commutative, the order in which we apply the actions does not matter. Since we are doing all additions modulo 2, applying the same action more than once is a no-op. This means that each possible action needs to be applied at most once. Let $x_i$ denote whether or not action $\textbf{a}_i$ was applied. Then we need to find a solution to:

$$\textbf{b} + x_1 \textbf{a}_1 + x_2 \textbf{a}_2 + x_3 \textbf{a}_3 + ... = \textbf{0}.$$

Since -1 = 1 in $\mathbb{Z}_2$, we can rewrite this as:

$$M \textbf{x} = \textbf{b},$$

where $M$ is the matrix of actions, in which column $i$ represents $\textbf{a}_i$, the action of toggling cell $i$:

$$M = \left[\begin{myarray} 1 & 1 & & & & 1 & & & & & & & & & & & & & & & & & & & \\ 1 & 1 & 1 & & & & 1 & & & & & & & & & & & & & & & & & & \\ & 1 & 1 & 1 & & & & 1 & & & & & & & & & & & & & & & & & \\ & & 1 & 1 & 1 & & & & 1 & & & & & & & & & & & & & & & & \\ & & & 1 & 1 & & & & & 1 & & & & & & & & & & & & & & & \\ 1 & & & & & 1 & 1 & & & & 1 & & & & & & & & & & & & & & \\ & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & & & & & & & & \\ & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & & & & & & & \\ & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & & & & & & \\ & & & & 1 & & & & 1 & 1 & & & & & 1 & & & & & & & & & & \\ & & & & & 1 & & & & & 1 & 1 & & & & 1 & & & & & & & & & \\ & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & & & \\ & & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & & \\ & & & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & & & & \\ & & & & & & & & & 1 & & & & 1 & 1 & & & & & 1 & & & & & \\ & & & & & & & & & & 1 & & & & & 1 & 1 & & & & 1 & & & & \\ & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & & \\ & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & & \\ & & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & & & & 1 & \\ & & & & & & & & & & & & & & 1 & & & & 1 & 1 & & & & & 1 \\ & & & & & & & & & & & & & & & 1 & & & & & 1 & 1 & & & \\ & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & & \\ & & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & \\ & & & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 \\ & & & & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 \end{myarray}\right] .$$

(I've omitted the zero entries to make it easier to see the structure of the matrix.)

We are now ready to answer some interesting questions about this game.

Are all boards solvable?

All possible boards are solvable if and only if $M \textbf{x} = \textbf{b}$ has a solution for all $\textbf{b}$, which is true if and only if $M$ has full rank. To find the rank of $M$, we convert it to its reduced row echelon form:

$$R = \left[\begin{myarray} 1 & & & & & & & & & & & & & & & & & & & & & & & & 1 \\ & 1 & & & & & & & & & & & & & & & & & & & & & & 1 & \\ & & 1 & & & & & & & & & & & & & & & & & & & & & 1 & 1 \\ & & & 1 & & & & & & & & & & & & & & & & & & & & 1 & \\ & & & & 1 & & & & & & & & & & & & & & & & & & & & 1 \\ & & & & & 1 & & & & & & & & & & & & & & & & & & 1 & 1 \\ & & & & & & 1 & & & & & & & & & & & & & & & & & & \\ & & & & & & & 1 & & & & & & & & & & & & & & & & 1 & 1 \\ & & & & & & & & 1 & & & & & & & & & & & & & & & & \\ & & & & & & & & & 1 & & & & & & & & & & & & & & 1 & 1 \\ & & & & & & & & & & 1 & & & & & & & & & & & & & 1 & \\ & & & & & & & & & & & 1 & & & & & & & & & & & & 1 & \\ & & & & & & & & & & & & 1 & & & & & & & & & & & & \\ & & & & & & & & & & & & & 1 & & & & & & & & & & 1 & \\ & & & & & & & & & & & & & & 1 & & & & & & & & & 1 & \\ & & & & & & & & & & & & & & & 1 & & & & & & & & 1 & 1 \\ & & & & & & & & & & & & & & & & 1 & & & & & & & & \\ & & & & & & & & & & & & & & & & & 1 & & & & & & 1 & 1 \\ & & & & & & & & & & & & & & & & & & 1 & & & & & & \\ & & & & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 \\ & & & & & & & & & & & & & & & & & & & & 1 & & & & 1 \\ & & & & & & & & & & & & & & & & & & & & & 1 & & 1 & \\ & & & & & & & & & & & & & & & & & & & & & & 1 & 1 & 1 \\ & & & & & & & & & & & & & & & & & & & & & & & & \\ & & & & & & & & & & & & & & & & & & & & & & & & \\ \end{myarray}\right] .$$

We find that $R$ has two zero rows, so $M$ has rank 23. This means there are unsolvable boards!

In fact, since we are in a finite field, we can count how many boards are solvable. A board is solvable if and only if it is in the column space of $M$. The fact that $M$ is symmetric and $R$ has two zero rows means that $M$ has two columns that are linearly dependent on other columns. After dropping those two columns, we are left with 23 linearly independent columns, i.e. 23 actions that cannot be obtained by combinations of other actions. Each of these actions can be applied 1 or 0 times, therefore we have $2^{23}$ solvable boards.

Since there are $2^{25}$ total possible boards, this means that only 1 in 4 boards is solvable. In other words, if the Brick Game generated a board by randomly sampling each cell, then that board would have a 75% chance of being unsolvable! The correct way to generate solvable boards is to start with an empty board, and randomly sample whether to apply each action 1 or 0 times.

(I still suspect that the Brick Game generated only solvable boards, and I was just too bad of a player to solve the more difficult ones. I don't have one of these devices here in the US, but if you have one and want to test this hypothesis, let me know!)

How to tell if a board is solvable? (attempt #1)

To solve a board $\textbf{b}$, we have to find a solution to $M \textbf{x} = \textbf{b}$, which we can do by Gaussian elimination. But if we only wanted to tell whether a board is solvable (without finding an actual solution), is there an easier way?

Let us call the set of all solvable boards $W$. We have already determined that $W = \text{col}(M)$. Since $W$ is closed under addition and scalar multiplication, it is a subspace of $\mathbb{Z}_2^{25}$. What if we tried to find its orthogonal complement, $W^\perp$? Then we could uniquely represent any board as $\textbf{b} = \textbf{w} + \textbf{w}^\perp$, where $\textbf{w} \in W$ and $\textbf{w}^\perp \in W^\perp$. And if $\textbf{w}^\perp$ is non-zero, then the board should be unsolvable.

Using the fundamental theorem of linear algebra, we get that $W^\perp = \text{null}(M^\top)$, which is the same as $\text{null}(M)$ since $M$ is symmetric. We can find a basis for $\text{null}(M)$ by manipulating the reduced row echelon form of $M$. We find that $\text{null}(M)$ is spanned by two vectors:

$$\textbf{n}_1 = [0 1 1 1 0 | 1 0 1 0 1 | 1 1 0 1 1 | 1 0 1 0 1 | 0 1 1 1 0],$$ $$\textbf{n}_2 = [1 0 1 0 1 | 1 0 1 0 1 | 0 0 0 0 0 | 1 0 1 0 1 | 1 0 1 0 1].$$

Here we encounter an apparent contradiction: $\textbf{n}_1$ is clearly solvable:

$$\textbf{n}_1 = \begin{bmatrix} & \blacksquare & \fbox{$\blacksquare$} & \blacksquare & \\ \blacksquare & & \blacksquare & & \blacksquare \\ \fbox{$\blacksquare$} & \blacksquare & & \blacksquare & \fbox{$\blacksquare$} \\ \blacksquare & & \blacksquare & & \blacksquare \\ & \blacksquare & \fbox{$\blacksquare$} & \blacksquare & \end{bmatrix}.$$

Where did we go wrong? It turns out that the concept of orthogonality doesn't really work in $\mathbb{Z}_2$. Two vectors are orthogonal if and only if their dot product is zero. In $\mathbb{R}^n$, the only vector that is orthogonal to itself is the zero vector. This means that for a subspace $V$ of $\mathbb{R}^n$, we have $V \cap V^\perp = \{\textbf{0}\}$, which enables the unique decomposition mentioned earlier. But in $\mathbb{Z}_2^n$, any vector that has an even number of 1s is orthogonal to itself, for example $[1 0 1 0 0]$. If such a vector is in some subspace $U$ of $\mathbb{Z}_2^n$, then it must also be in $U^\perp$, so $U \cap U^\perp \neq \{\textbf{0}\}$, and we lose the unique decomposition property.

How to tell if a board is solvable? (attempt #2)

Since the fancy fundamental theorem didn't help us here, let's go back to the basics. We know that a board $\textbf{b}$ is solvable if and only if it is in $\text{col}(M)$, which is the same as $\text{row}(M)$ since $M$ is symmetric. Converting a matrix to its reduced row echelon form does not change its row space. This means that a board is solvable if and only if it is in $\text{row}(R)$, which is the same as $\text{col}(R^\top)$. In other words, a board is solvable if and only if $R^\top \textbf{y} = \textbf{b}$ has a solution. Let's take a closer look at $R^\top$:

$$R^\top = \left[\begin{myarray} 1 & & & & & & & & & & & & & & & & & & & & & & & & \\ & 1 & & & & & & & & & & & & & & & & & & & & & & & \\ & & 1 & & & & & & & & & & & & & & & & & & & & & & \\ & & & 1 & & & & & & & & & & & & & & & & & & & & & \\ & & & & 1 & & & & & & & & & & & & & & & & & & & & \\ & & & & & 1 & & & & & & & & & & & & & & & & & & & \\ & & & & & & 1 & & & & & & & & & & & & & & & & & & \\ & & & & & & & 1 & & & & & & & & & & & & & & & & & \\ & & & & & & & & 1 & & & & & & & & & & & & & & & & \\ & & & & & & & & & 1 & & & & & & & & & & & & & & & \\ & & & & & & & & & & 1 & & & & & & & & & & & & & & \\ & & & & & & & & & & & 1 & & & & & & & & & & & & & \\ & & & & & & & & & & & & 1 & & & & & & & & & & & & \\ & & & & & & & & & & & & & 1 & & & & & & & & & & & \\ & & & & & & & & & & & & & & 1 & & & & & & & & & & \\ & & & & & & & & & & & & & & & 1 & & & & & & & & & \\ & & & & & & & & & & & & & & & & 1 & & & & & & & & \\ & & & & & & & & & & & & & & & & & 1 & & & & & & & \\ & & & & & & & & & & & & & & & & & & 1 & & & & & & \\ & & & & & & & & & & & & & & & & & & & 1 & & & & & \\ & & & & & & & & & & & & & & & & & & & & 1 & & & & \\ & & & & & & & & & & & & & & & & & & & & & 1 & & & \\ & & & & & & & & & & & & & & & & & & & & & & 1 & & \\ & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & 1 & & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & & \\ 1 & & 1 & & 1 & 1 & & 1 & & 1 & & & & & & 1 & & 1 & & 1 & 1 & & 1 & & \\ \end{myarray}\right] .$$

Since the last two columns of $R^\top$ are zeros, the entries $y_{24}$ and $y_{25}$ can take any values. And since the first 23x23 block of $R^\top$ is an identity matrix, we can read the first 23 entries of $\textbf{y}$ directly from $\textbf{b}$:

$$y_1 = b_1,~ y_2 = b_2,~ ...,~ y_{23} = b_{23}.$$

The last two rows of $R^\top$ determine whether $\textbf{b}$ is solvable or not. In particular, $\textbf{b}$ is solvable if and only if both of the following conditions hold:

$$\begin{align*} b_{24} =~ & b_2 + b_3 + b_4 + b_6 + b_8 + b_{10} + b_{11} + b_{12} + \\ & b_{14} + b_{15} + b_{16} + b_{18} + b_{20} + b_{22} + b_{23}, \\ b_{25} =~ & b_1 + b_3 + b_5 + b_6 + b_8 + b_{10} + b_{16} + b_{18} + b_{20} + b_{21} + b_{23}. \end{align*}$$

Let's simplify them a bit:

$$\begin{align*} b_1 + b_2 + b_4 + b_5 + b_{11} + b_{12} + b_{14} + b_{15} + b_{21} + b_{22} + b_{24} + b_{25} &= 0 \\ b_1 + b_3 + b_5 + b_6 + b_8 + b_{10} + b_{16} + b_{18} + b_{20} + b_{21} + b_{23} + b_{25} &= 0 \end{align*}$$

We can visualize these two conditions as follows:

$$\textbf{c}_1 = \begin{bmatrix} + & + & & + & + \\ & & & & \\ + & + & & + & + \\ & & & & \\ + & + & \phantom{+} & + & + \end{bmatrix},~~~ \textbf{c}_2 = \begin{bmatrix} + & & + & & + \\ + & & + & & + \\ & & & & \\ + & & + & & + \\ + & \phantom{+} & + & \phantom{+} & + \end{bmatrix}.$$

For a board to be solvable, the cells marked with $+$ must add up to zero. In other words, the locations marked with $+$ must have an even number of "on" cells, for both $\textbf{c}_1$ and $\textbf{c}_2$. This gives us a way to tell whether a board is solvable, without even needing to use pen and paper!

You can think of $\textbf{c}_1$ and $\textbf{c}_2$ as checksums, or parity codes. Interestingly, neither of them checks the value of cells $b_7$, $b_9$, $b_{13}$, $b_{17}$, or $b_{19}$, which means that a board is solvable regardless of its values in those cells.

Note that although $\textbf{y}$ can be read off from $\textbf{b}$, it is not a solution to the original system $M \textbf{x} = \textbf{b}$. The columns of $R^\top$ are linear combinations of the columns of $M$, so we have to do more work to find an actual solution for a given board.

How to solve a board?

As mentioned earlier, we can find a solution to $M \textbf{x} = \textbf{b}$ by Gaussian elimination. But is there an easier way? In particular, is there a linear operation that would map every board to one of its solutions?

Because $M$ is not full rank, and because we are in $\mathbb{Z}_2$, the usual methods for computing a (pseudo)inverse don't work.

Let's take another look at the process by which we converted $M$ to its reduced row echelon form $R$. Each row of $R$ is a linear combination of rows of $M$. By keeping track of which rows we added together to obtain $R$, we can express $R = K M$, where $K$ is another 25x25 matrix in $\mathbb{Z}_2$.

Then, if we left-multiply $M \textbf{x} = \textbf{b}$ by $K$, we get $R \textbf{x} = K \textbf{b}$. Let $\textbf{d} = K \textbf{b}$, and let $\textbf{v}_1$ and $\textbf{v}_2$ denote the last two columns of $R$. Then $R \textbf{x} = \textbf{d}$ has the following form:

$$\begin{bmatrix} I_{23} & \textbf{v}_1 & \textbf{v}_2 \\ \textbf{0} & 0 & 0 \\ \textbf{0} & 0 & 0 \end{bmatrix} \begin{bmatrix} \textbf{x}_{1:23} \\ x_{24} \\ x_{25} \end{bmatrix} = \begin{bmatrix} \textbf{d}_{1:23} \\ d_{24} \\ d_{25} \end{bmatrix}.$$

From here, we can immediately tell that if $d_{24}$ or $d_{25}$ are non-zero, then the board is unsolvable. If it is solvable, then one possible solution is:

$$ x_1 = d_1,~ x_2 = d_2,~ ...,~ x_{23} = d_{23},~ x_{24} = 0,~ x_{25} = 0. $$

Thus, the matrix $K$ gives us what we wanted. For any board $\textbf{b}$, we can look at the last two entries of $K \textbf{b}$. If those entries are zeros, then the board is solvable, and we can read off the solution from the first 23 entries of $K \textbf{b}$.

Note that a solution obtained this way might be "unnatural," because it never uses the 24th or 25th action ($x_{24} = x_{25} = 0$). For example, the following board is solvable by a single application of the 25th action:

$$\begin{bmatrix} \phantom{\blacksquare} & \phantom{\blacksquare} & \phantom{\blacksquare} & & \\ & & & & \\ & & & & \\ & & & & \blacksquare \\ & & & \blacksquare & \fbox{$\blacksquare$} \end{bmatrix},$$

but the method above finds a much more circuitous solution:

$$\begin{bmatrix} \fbox{$\phantom{\blacksquare}$} & \phantom{\blacksquare} & \fbox{$\phantom{\blacksquare}$} & \phantom{\blacksquare} & \fbox{$\phantom{\blacksquare}$} \\ \fbox{$\phantom{\blacksquare}$} & & \fbox{$\phantom{\blacksquare}$} & & \fbox{$\phantom{\blacksquare}$} \\ & & & & \\ \fbox{$\phantom{\blacksquare}$} & & \fbox{$\phantom{\blacksquare}$} & & \fbox{$\blacksquare$} \\ \fbox{$\phantom{\blacksquare}$} & & \fbox{$\phantom{\blacksquare}$} & \blacksquare & \blacksquare \end{bmatrix}.$$

Furthermore, the matrix $K$ that row-reduces $M$ is rather ugly:

$$K = \left[\begin{myarray} & & & & & 1 & & & & & 1 & 1 & & & & 1 & & 1 & & & & 1 & 1 & 1 & \\ & & & & & & 1 & & & & 1 & 1 & 1 & & & & & & 1 & & 1 & 1 & & 1 & 1 \\ & 1 & 1 & & & 1 & & 1 & 1 & & 1 & 1 & 1 & & 1 & 1 & 1 & 1 & 1 & 1 & 1 & & & 1 & 1 \\ & & 1 & 1 & 1 & & 1 & & & & 1 & 1 & & 1 & 1 & & 1 & & & & & & 1 & 1 & 1 \\ & 1 & 1 & & 1 & 1 & & & & & 1 & & 1 & & 1 & & & 1 & & 1 & 1 & 1 & & & \\ & 1 & & 1 & 1 & 1 & 1 & & & & 1 & 1 & 1 & 1 & 1 & 1 & & 1 & 1 & & & 1 & 1 & 1 & \\ & 1 & & 1 & & 1 & 1 & & 1 & 1 & & & & 1 & & 1 & 1 & 1 & & & & 1 & & & \\ & & & & 1 & & & & 1 & 1 & & & & & 1 & & & & & & & & & & 1 \\ & 1 & & 1 & & 1 & 1 & & 1 & 1 & & 1 & & & & & & 1 & 1 & 1 & & & & 1 & \\ & 1 & & 1 & 1 & 1 & 1 & & & & & 1 & 1 & 1 & & & 1 & 1 & & 1 & 1 & 1 & 1 & 1 & 1 \\ & & & & 1 & & & & 1 & 1 & & & 1 & & 1 & 1 & 1 & 1 & 1 & & & 1 & & & \\ & & & & & & & & & & & & & & & & 1 & & & & 1 & 1 & 1 & & \\ & 1 & 1 & & 1 & 1 & & & & 1 & 1 & & 1 & 1 & & & & 1 & & & 1 & 1 & & & \\ & & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & 1 & & & & & & 1 & & 1 & 1 & & 1 & 1 \\ & 1 & 1 & & & 1 & & & 1 & & 1 & & & 1 & 1 & & 1 & 1 & 1 & 1 & & & 1 & & 1 \\ & 1 & & 1 & & 1 & 1 & & 1 & 1 & & 1 & & 1 & & & & & & & 1 & 1 & & 1 & \\ & & 1 & 1 & & & 1 & & & 1 & 1 & 1 & & & 1 & & 1 & 1 & 1 & & & & 1 & & \\ & 1 & & 1 & 1 & 1 & 1 & & & & & 1 & 1 & 1 & 1 & & 1 & 1 & 1 & & 1 & 1 & 1 & 1 & \\ & 1 & 1 & & & 1 & & & 1 & & 1 & & & 1 & 1 & & 1 & 1 & 1 & & & & 1 & & \\ & & & & & & & & & & & & & & & & & & & & & & & & 1 \\ & 1 & 1 & & 1 & 1 & & & & 1 & 1 & & 1 & 1 & & & & & & & 1 & & 1 & 1 & \\ & & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & 1 & & & & & & & & 1 & 1 & 1 & & \\ & 1 & 1 & & & 1 & & & 1 & & 1 & & & 1 & 1 & & 1 & 1 & 1 & & & & 1 & 1 & \\ & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & 1 & & 1 & 1 & 1 & & 1 & & 1 & & 1 & 1 & 1 & \\ 1 & 1 & & 1 & 1 & & & & & & 1 & 1 & & 1 & 1 & & & & & & 1 & 1 & & 1 & 1 \\ \end{myarray}\right] .$$

We can make a few simplifications to it. For example, we can replace the last two rows with zeros, since the last two rows of $R$ are zeros. We can also add $\textbf{n}_1$ or $\textbf{n}_2$ (or both) to any row of $K$, while preserving the property that $R = K M$. But even after these attempts to simplify $K$, it still remains pretty dense. So even though it gives us a way to solve any board with a single linear operation, it is not easy to find that solution without pen and paper. The good news is that the game remains fun to play by trial and error.

This is as far as I got when trying to apply linear algebra to this game. If you have any further insights, I'd love to hear about them.

Special thanks to YL for helpful suggestions and feedback. Supporting code for this article can be found here.

2018 Q1 reads

2018-04-14T00:00:00-07:00

Some predictions by Rodney Brooks about self-driving cars and AI / ML in general.
A talk by Maciej Ceglowski on the history of radio, which started as a nerds' toy but eventually became a political tool. He argues that the Internet is going through the same transition today.
A Quillette article drawing some connections between the tech industry and the autism spectrum.
A Cracked article reminding readers that the world doesn't care if you're a nice guy; it only cares about what it can get from you.
Two of the more reasonable responses to the Aziz Ansari debacle.
A podcast by More Perfect providing some nuance on the Supreme Court's Citizens United decision.
A long article by Michal Zalewski on preparing for disasters of various kinds. This is the security researcher who wrote The Tangled Web, and it's illuminating to see how he thinks about security in the physical world.

Never Sell

2018-02-25T00:00:00-08:00

Dust hangs in the feeble yellow light of the bare bulb in the attic. Around me are towers of boxes, stacked magazines, a folding bed, and some suitcases that would look more at home in a black-and-white movie. The residue of decades in the lives of people I know nothing about, except that they were willing to part with this house for a curiously modest sum.

My job is to sort this junk into two piles: garbage and garage sale. Where do I even start? I walk over to the only window and pull at the shades, trying to get them unstuck. Something falls to the floor with a thud. I pick it up and wipe off the dust with my sleeve. Whatever this is, someone made sure to wrap it in a lot of brown packing tape. It's about the size of a paperback, but stiffer, and its weight is uneven. One side is completely discolored; it must have been facing the window. And on that side, with a pen whose ink has almost faded but whose indentations still remain, someone scribbled "NEVER SELL."

I lift it closer to my eyes trying to discern any other markings, but the dust gets to me and I'm caught in a savage fit of sneezing. It's a miracle that I don't lose a couple of limbs as I stumble down the attic stairs, through the freshly vacuumed hallway, and into the newly IKEA-d living room.

"What's wrong?" asks Carrie, her forehead rising with that look of concern that is so uniquely hers. I remember glimpsing that expression on the day we met, and three years later it still makes me feel like a knot inside my chest is coming undone.

I lift the box towards her and try to speak, but as soon as I take a breath, the sneezing returns and knocks it right out of me. My vision now blurry, I drop the box on the coffee table and scamper to the bathroom to wash the dust off.

When I return, Carrie is sitting on the sofa with a puzzled look on her face. Her fingers trace over the faded inscription.

"Coke," she decides. "Was there a suitcase with money, too? And some big guys with expensive suits, black shades, and machine guns?"

"You watch too many bad movies," I reply, my voice raspy from all the sneezing.

She lifts the box to her ear, hears it rattle, decides that the powder hypothesis is unlikely. I curl up on the couch by her side, nestling my head on her lap, right next to her swollen belly. She looks down at me and smiles, then turns my forehead a few degrees so she can balance the box on top. It feels cool to the touch.

"Maybe we should call Mr and Mrs Rodahl?" she asks quietly.

I raise one eyebrow in a "Do we?" expression. The box almost falls, but she catches it.

"I mean, they said we could toss out whatever's left in the attic, that it would save them a trip to the landfill," she muses. "But what if they forgot something really important? Something so valuable that they would never sell it or throw it away?"

I raise my other eyebrow, and the box teeters off balance again. She lifts it off my forehead and puts it back on the coffee table.

"Don't you want to know what's inside?" she teases as her fingers gently push my eyebrows back into their normal position.

"Curiosity killed the cat," I quip.

"And satisfaction brought it back!" she laughs.

"Yeah, but only eight times."

"How many do you think we have left?"

"I don't know; I'd have to check my cat diary."

"Are you sure it's not a lab notebook?"

"I wouldn't know; it's written in cattish."

"Well, does it have diagrams?"

"Some fishbones here and there."

"Is your cat named Ishikawa or something?"

"Okay, you win."

In full dimple mode, she watches me as I shuffle to the kitchen, open five wrong drawers, and return at last with a paring knife. The box is on the coffee table in front of us, its secrets not safe for much longer.

After a careful cut, the layers of packing tape start peeling away with an indignant sound. Carrie is holding her breath. Where the tape ends, there's cardboard. I tear open one side with my fingers and see a row of smooth black rectangular buttons. Above them are six small symbols that I remember from childhood, when I knew their order without having to look: Record, Play, Rewind, Fast Forward, Stop/Eject, and Pause. Man, I haven't seen one of these in years.

I pull the cassette recorder out of the box and place it gently on the table. There's a tape inside. Carrie and I look at each other. She presses the Play button, but nothing happens. After turning the device around and fiddling with its back, she reveals an empty battery compartment.

"Do we even have double As in the house?" I ask.

Carrie produces two out of the TV remote, then walks over to the kitchen counter and takes two more from the carbon monoxide alarm. She hits Play again. There's a scratching sound, and then the tentative whimper of a violin being tuned. After a few false starts, the room fills with the unmistakeable opening notes of Iron Lady's Caprice in Four Strings. Except it's broken somehow; it climbs to a climax and then stumbles, drops off a steep ledge, shatters, recovers, restarts. It has all the components of the masterpiece, but none of its effortless grace.

"Iron Lady," Carrie whispers. "Imagine that. Didn't the Rodahls say something about a daughter? They seemed skittish when I asked them."

We listen in silence for a few minutes. The piece ends. I rewind the tape and press Play again.

"Her real name is Ellen Rodahl," I recall, looking up at the walls and the wooden ceiling with newfound appreciation. "This could be the house she grew up in. I wonder why they never told us." The gears in my head spin like a slot machine and land on the jackpot. "Do you know how much the Wessons paid for the recording of her first concerto? Of course it's nothing to them; they're billionaires."

Carrie frowns. "They're completionists. They want to own Iron Lady and everything she's ever created. It was in the Post last week. They even bullied her middle school into selling them the tapes from her fifth grade recital."

"Sounds like their complete collection is not so complete."

My throat feels dry, like when my dad and I talked about money the day I left for college. I swallow hard. Carrie sees the dollar signs in my eyes and looks unhappy.

"We should really call the Rodahls," she says.

"You know, this tape could pay off the house! We could even have a second baby! And take that trip to Scotland that we've been postponing! Just think about-"

"You want to sell this thing?" Carrie cuts me off, her voice like cold water. "It's not even ours to begin with."

"It's ours now that we bought the house! They left us all the crap in the attic. We have the paperwork; they have no claim to it now."

"You want to sell a piece of her childhood, something her parents wanted to hide from the world and keep for themselves? You want to sell it to some psychopathic billionaire sons-of-bitches with a fetish for adolescent violin-playing girls?"

I open my mouth to speak, then pinch my lips back together and try to remember what I learned about conflict resolution. Take a deep breath, step back, find common ground...

"We don't have to decide this today. Let's sleep on it and see how we feel about it tomorrow."

I force myself to relax and nudge closer to Carrie, blinking slowly at her like a peace-loving cat. Her face softens as I put my arms around her. These days when I do that, I can barely reach one hand with the other. She is soft and warm and most of all mine, and I must never forget how lucky I am.

The fridge in the kitchen clatters to life and fills the silence with an insistent rumble. I must've been so distracted, I didn't even notice the Caprice playing a second time. Carrie pushes the Stop button and a little red light goes off. She rewinds the tape and hits Play again. There's a scratching sound and a click.

"Her real name is Ellen Rodahl," the tape says. "This could be the house she grew up in."

Rest-of-2017 Reads

2017-12-31T00:00:00-08:00

Books:

Available, by Matteson Perry -- an eye-opening account of what dating can be like for a guy who is good at it.
Modern Romance, by Aziz Ansari -- not particularly insightful, but the chapter on Japan was fascinating.
American Hookup, by Lisa Wade -- a sociologist's take on the phenomenon that caused a large part of my culture shock when I first came to the US.
The Happiness Hypothesis, by Jonathan Haidt -- a remarkable synthesis of what ancient philosophy and modern psychology can tell us about happiness. There are very few books that have made me stand in awe at the author's intellect; this is one of them. It was also a nice follow-up to a book I've read previously about Stoicism.
Atlas Shrugged, by Ayn Rand -- one of the best dystopias I've read so far, excluding the three-hour speech towards the end, and the unfortunate association that this book has with right-wing politics in the US. It depicts a world descending into collectivism and state control of the economy. What resonated with me was its clairvoyant portrayal of what happened in the USSR, and the uncanny parallels with what is happening today in Venezuela.

Cool stuff online:

A Planet Money / Rough Translation episode on the role of "fake news" in the war in Ukraine.
A surreal, Black Mirror-like Atlantic article on hiring fake friends and family in Japan.
An optimistic look at the future of electric and autonomous transportation. Unfortunately, oil is used for lots more things than gasoline.
A Scott Alexander piece on sexual harassment and how it is handled differently depending on the victim's gender. It seems that every time a gender issue explodes into public debate in ways that I find deeply troubling, Scott Alexander provides a rare voice of reason in that debate.
Another Scott Alexander article on how people can have completely different experiences of interactions with other people, even though they have the same demographics and geography.

Biking the CA-1 Island

2017-10-05T00:00:00-07:00

One of the best days of my life.

Up at 6:30, out by 7. Sun rising from the hills as I drove south on 101. Ecuador-grade fog over the hills to Monterey, but gone by Big Sur. Quick hike-a-bike over a dusty trail, and then I was on The Island (the 30-mile stretch of Highway 1 that was cut off from car traffic by a broken bridge to the north and a landslide to the south).

The road was windy, and delightfully empty, with postcard views around every corner. I got to Gorda, drank some chocolate milk, and headed back north. With lots of daylight still left, I stopped at an unmarked trail and found my way to the ocean. Powerful waves crashed against the cliffs, exploding in outbursts of clean white foam. I could have watched them for hours. It was just warm enough in the sun not to need a jacket.

A rare wave of gratitude hit me as I sat there. Epiphany #1: I do in fact have a lot to lose, even though I'm single / unhappy / whatever. Epiphany #2: This body that allows me to do things like this, I'm in fact very thankful for it, and I should take better care of it so that I can continue doing things like this for as long as possible.

(more pictures)

Sequoia

2017-09-20T00:00:00-07:00

Cotton candy clouds
Extreme lightning danger
First solo trip to a national park.

(more pictures)

June, July, and August Reads

2017-09-13T00:00:00-07:00

I've been on a cycling / hiking / camping binge this summer, and haven't read as much as I usually do. Italo Calvino's If on a Winter Night a Traveler was a fun twist on narrative structure. Scott Alexander had a good article on gender differences. Radiolab had a great episode on free will and justice (favorite quote: "Free will is the biology we don't understand yet"), and another great one the technological future of fake news. And Danny MacAskill made the most amazing love song to Scotland that I have ever seen (look up the behind-the-scenes video too).

Crater Lake

2017-07-10T00:00:00-07:00

Snow in the middle of summer
Inverted skies
A mirror for the ages.

(more pictures)

March, April, and May Reads

2017-06-01T00:00:00-07:00

Salman Rushdie's novel Two Years Eight Months and Twenty-Eight Nights was entertaining and surprisingly layered.
lcamtuf's fascinating blog post on generating valid JPEG files using a fuzzer.
An older, first-person Wired piece about cochlear implants.
This American Life on testosterone and Putin.
Radiolab on nukes and jury nullification.

The Waste that Dreams Realized Leave Behind

2017-03-08T00:00:00-08:00

Some people are born in the wrong place.

I can't pinpoint the exact moment when my love affair with the United States started, but I know it was pretty early on. I still have notebooks from middle school where I wrote essays about wanting to go study in the US, see the Statue of Liberty in the city where the buildings scraped the sky, and other clichéd images for which I can only thank Hollywood's global propaganda machine.

My early obsession with the US was unapologetically materialistic. Growing up, I always had food on the table, but my family was far from being well off. The breakup of the Soviet Union and the subsequent currency devaluation wiped out my parents' savings, and things were tough for a while. But I was too little to remember, and things got better by the time my age hit two digits. We were a family of four squeezed into a tiny two-room apartment, we never had a car, and we ate out only about once or twice per year. But at least we had stability and peace, and my parents were wise enough to make sure that my brother and I did well in school.

My classmates and I all believed that leaving the country was the path to a better life, and everyone who had a chance to escape did so. I remember an old friend calling me from Italy, where his family had moved when we were in middle school, and casually mentioning a game console that he had, or some such object of lust that I envied with the kind of intensity that only a teenager can muster. I also remember watching Cartoon Network (the only TV channel that aired American ads without replacing them with local ones), and drooling over the RC racing cars that I thought every American kid had.

And then there were the PC magazines that my uncle (who ran a computer store in the city) brought from a trade show that he attended in the US. I spent hours pouring over the specs of the latest computers (400 MHz! OMG!), and I cultivated a multi-year obsession with Palm PDAs. Oh Sony CLIÉ PEG-SJ22, how you have haunted my dreams... I even built a cardboard enclosure to house the future you, and downloaded a Palm OS simulator on my dad's computer trying to get to know you. But you were not to be mine. There was no place in the country where I could procure you, and even if there had been, your $199 price tag was out of reach.

Anyway, I digress.

Some people are born in the wrong place. As a teenager, I resented being born in Moldova. I was convinced that happiness meant owning all those coveted electronic gadgets, and that I could have them if only I were in the US instead. I'm still not sure why I felt pulled to the US in particular, and not to other rich countries in Western or Northern Europe. The German Dream and the French Dream don't sound quite the same as the American Dream, do they? I suspect this had to do with learning English in school (which was a natural choice for a kid obsessed with computers), and with being exposed to more American culture. For example, I somehow got my hands on electronic versions of Tony Robbins's Personal Power and Robert Kiyosaki's Rich Dad, Poor Dad when I was 15 or 16. (Thank you Internet!) Those seem silly now, but I found them hugely inspirational at the time. They were a promise of the personal triumph and financial success that were possible in America, even though there was little in them that I could apply as a high school student in a third-world backwater.

By the time we were in high school, my classmates and I purposefully plotted our way into US colleges. One path was to pay, and the other path was to get a scholarship. The former was a non-starter for many of us, so we buckled down for the latter. We did everything we could to get awards in Math, CS, or Physics. We studied for our standardized tests at the EAC, a local resource sponsored by the Soros Foundation. (Thank you billionaire class!) I even had to cross the border into Romania to take my SATs before the applications were due.

In the end, my applications were less than stellar. I did not have the kind of international olympiad medal that got some of my classmates into Harvard or MIT. Of the ten applications that I submitted, my final tally was rejected (5), waitlisted and ultimately rejected (4), and accepted (1: Tufts University). The latter was partly due to a special scholarship that they had for one or two Eastern Europeans, the Boryana Damyanova scholarship, which I didn't even know about when I was applying. (Thank you one-percenters, again.)

It's hard to overstate how lucky I got with my Tufts acceptance. It could have easily gone the other way, and then my life's trajectory would have been completely different. Of course, going to college is not the only way into the US, and I know people now who arrived later and are none the worse for it. And maybe I would've been just as happy, or happier, at a university somewhere in Europe. But back then, going to study in the US had been my dream for so long that I hadn't even considered other options, and it would've been an epic defeat if I hadn't gotten in.

So in August of 2008, I said goodbye to my parents and got on the plane to Boston. It felt surreal, like an out-of-body experience. Some of the first things I remember after landing: the smell of the ocean, the cries of seagulls, and the taxi driver who asked me: Какой родной язык у тебя? (My accent was apparently close enough to Russian, even though I speak Romanian at home.)

I've come a long way since that first day on US soil, and the adjustment has been rocky at times. It took me a year or two to become really comfortable with the language, another few years to learn how to make friends, and even longer to stop sucking at dating. Only in the last year or two did I gain some confidence that I can build for myself the kind of life that I want, and I'm not sure if that protracted process of growing up would've unfolded faster if I hadn't decided to switch continents in the middle.

This story should end here, with a shot of me walking into the sunset, and an American flag waving somewhere in the background, right? But this post is not called "The Taste of Dreams Realized"; it's called "The Waste that Dreams Realized Leave Behind." (Both are lines from the song Decade and One by Vienna Teng.) Because my sense of rightness about being in the US, which I've been so convinced of since middle school, never fully materialized once I arrived. And even now, after spending almost a third of my life here, I'm still not sure whether I want to stay for the long term.

My utopian vision of the US as just, fair, and safe underwent a series of violent readjustments as I learned more about the country's history, politics, race relations, foreign and immigration policy, ubiquitous surveillance, etc. For most of my time here, first at Tufts and then at Berkeley, I've been sheltered in suburban campus bubbles. But for the last year or so, I've been going to work in downtown San Francisco, which is accurately described as an outdoor crack house and mental asylum, and represents the latest adjustment to my understanding of what life in US cities is really like. I try to take my own greener-grass thinking with a grain of salt, and I'm sure I'd find things to complain about wherever I go, but I have trouble imagining myself settling in the Bay Area. The things holding me here are my circle of friends and the good jobs in Silicon Valley, not any particular liking for the place itself.

Another reason for my perpetual doubts about staying in the US is that my ability to stay here is not fully within my control. I've been on a student visa this entire time, and the path to a work visa and then to permanent residency requires a bit of luck and a lot of waiting. There is no guarantee that it will work out, since the H-1B system is a lottery. I have to scratch my head when a country offers me two world-class degrees fully paid for, allows me to get a great job in Silicon Valley, but then makes it so difficult to stay here long term. It's hard to commit to a place when they can kick you out on a whim. (And yes, I recognize that Muslims and others have it much worse.)

But even if I learned to accept the US as the imperfect place that it is, and if I managed to secure a green card eventually, I'd still have to wrestle with the fragmented identity that is the curse and blessing of every immigrant. No amount of catching up on American pop culture will close the rift of understanding that exists between me and someone who grew up here reciting the Pledge of Allegiance. Similarly, no amount of study abroad will allow an American to experience growing up in a third-world country. So there will always be this sense of otherness that prevents me from becoming fully American, even if that's more achievable than becoming, say, Swiss or German. Unsurprisingly, my closest friends are fellow immigrants and Americans who have spent significant time abroad and have a non-US-centric perspective.

On the other side, I can't count myself as fully Moldovan / Romanian, either. A big chunk of my growing up -- learning how to drive, how to file my taxes, how to interview for a job, how to ask someone on a date -- happened here in the US, in English. Going back "home" feels like being 18 again, and having to relearn all those things. It is the opposite of empowering. On top of that, living in a multicultural society and in the center of the tech world means that my values and interests have diverged quite a bit from the things that people back home care about. Religion and patriotism are examples of things that are important to people there, but that I think are mostly harmful. When interacting with ex-teachers, ex-classmates, etc. I feel the same chasm in perspective between us, the same sense of otherness, like I am a foreigner back "home," too. The only Moldovans I keep in touch with, apart from my family, are folks who have left the country as well.

I have one foot in one world and one foot in another, and I can't decide if I want to commit both feet to either side, or jump for a third one. The glass-half-full way to put it is that I am a global citizen, equally capable of making a home for myself anywhere in the world. The glass-half-empty view is that I am not adaptable but rootless, and that my sense of belonging to a local community will always be tentative at best. But maybe rootlessness and anxiety are just the price we pay for social mobility in today's world.

When people call me brave for leaving everything behind and moving to the US, I think they misunderstand. To paraphrase Hanlon's razor, never attribute to courage that which is adequately explained by youthful recklessness. At 18, I had an obsession and I followed it, consequences be damned. It was an easy decision. I didn't think that opening certain doors would cause others to close, and I didn't think anything about putting 7 and later 10 time zones between myself and my family. In some sense I'm glad for that lack of forethought, because I'm pretty happy with the shape that my life has taken so far, and if I'd done a more careful cost-benefit analysis, I might have been too scared to jump.

But the tradeoffs of switching countries are real, and paralyzing, when considering whether to settle here or attempt a second jump. My only decision so far has been to postpone any decision, and enjoy the good things that I have here for now (friends, job, familiarity with the language and the culture). Of course, indecisiveness has its own risks, and I would love to reach the point where I consciously commit to a place, instead of remaining there by inertia. It hasn't happened yet. But that's a terrible reason to stay indoors and ruminate, when I could instead go for a bike ride and (to use one of my favorite English expressions that doesn't mean what you think it means) put the balls to the wall.

Credits for the images: Ice Age (fair use)

February Reads

2017-03-03T00:00:00-08:00

The things Peter Welch sends to recruiters. I stumbled upon his site a few years ago, when his post Programming Sucks made the rounds on the Internet. The guy is a designer and programmer by day, but he moonlights as a writer, which I really admire. I think his sense of humor is excellent. (If you liked Programming Sucks, you might also like this unrelated Quora post about agile programming.)
Maciej Cegłowski's talk on website obesity. I recommend clicking through to the video, because he's a great speaker and he will make you laugh. The guy built a one-man bookmarking service that brings in enough money to not require a day job, which I think is really cool. I don't know if he'd be able to write and speak so freely if he were someone's employee.
Two very long articles about male nerds, the "nice guy" syndrome, and the underbelly of feminism. I came across them by sheer luck when a friend mentioned them about two years ago. I was in a pretty dark place back then, and I credit reading these (among other things) for allowing me to climb out. I recently re-read them while writing a letter to a friend, and they are every bit as powerful as I remembered.
The book Guns, Germs, and Steel, by Jared Diamond. I never liked history in school, because my teachers just wanted me to memorize dates and facts. This book offers a much more inspiring view of history as a way to understand the causes and effects that got us to the world we have today. Why did the Fertile Crescent and China lose their early lead in the development of advanced societies? Why did Europeans colonize North America and not the other way around? In a society organized as a nation-state, is kleptocracy the norm or the exception?
A Wired article about abusing slot machines that rely on insecure pseudorandom number generators.
A Planet Money episode about the surprising role of bees in today's intensive agriculture.

November, December, and January Reads

2017-01-30T00:00:00-08:00

Podcasts:

A Planet Money episode that takes a peek inside the fake-news industry.
A 99% Invisible episode on the surprising role the postal service played in early US history.
Crimetown, a fascinating new series about corruption and mafia in Providence a few decades ago.

Cool stuff online:

Two bizarre articles about automatism and sleepwalking as legal defenses. I learned about these from the book "The Power of Habit", by Charles Duhigg. From the same source I learned about Arbitron, a company with ingenious techniques for measuring the audience size of a radio or TV station. (These items of curiosity are only tangentially related to the topic of the book.)
A New Yorker article about the Mosul dam in Iraq. If they stop pumping cement into the earth, the dam collapses. Talk about technical debt.
Some incredible 3D printed heart gears, and an automated transmission model that includes a reverse gear.

September and October Reads

2016-10-30T00:00:00-07:00

Books:

Bright-sided was an entertaining look at what happens when positive thinking gets applied in delusional amounts. The tagline is: "To complainers everywhere: Turn up the volume!"
Extremely Loud and Incredibly Close was more creative and more touching than I expected. Favorite scenes: Typing one's life story on a typewriter with no ink, and the story about the Sixth Borough.
A Guide to the Good Life was an accessible introduction to Roman Stoicism. It's surprising how much this ancient philosophy has in common with the modern CBT approaches described in "Feeling Good" by David Burns.

Cool stuff online:

About the monorepo at Google. AKA: everything you think you know from working in startups is wrong.
The badass insanity that is /r/churning.
Mystery signal from a helicopter -- an illuminating peek at what a good signals analyst can do.
Companies in Silicon Valley working on flying cars. I had no idea this was happening. Also, check out this incredible video about pedal-powered flight.
Some really amazing and unusual designs for wooden clocks and other mechanisms.

August Reads

2016-09-01T00:00:00-07:00

Books:

Evicted by Matthew Desmond, an eye-opening account of life for renters in the US whose only source of income is social security. The most depressing thing is that this level of poverty (and lack of education / skills / jobs / stable housing) seems self-perpetuating, and there is little chance for people to climb out of it even if they are willing to work.
So Good They Can't Ignore You by Cal Newport, a counter-intuitive approach to the problem of finding work you love.
The 4-Hour Workweek by Tim Ferris, something I skimmed many years ago, and wanted to revisit. The level of independence that he has achieved is enviable, but I don't completely buy the "you can do this too in N easy steps" aspect of the book. Also, as a third-worlder, I find the way he talks about hiring foreign "virtual assistants" (and paying them peanuts) incredibly entitled.

Podcast episodes:

Two episodes from This American Life about the refugee camps in Greece. (Related: A long-form NYT Magazine piece about the war in Iraq and Syria and how it got to be the mess that it currently is. Reading a synthesis like this is so much more informative than reading the latest news.)
The Buried Bodies Case from Radiolab, an ethical dilemma of epic proportions, involving the attorney-client privilege.
Playing God from Radiolab, another ethical dilemma about distributing scarce resources during an emergency.

August Reads

2016-09-01T00:00:00-07:00

Books:

Evicted by Matthew Desmond, an eye-opening account of life for renters in the US whose only source of income is social security. The most depressing thing is that this level of poverty (and lack of education / skills / jobs / stable housing) seems self-perpetuating, and there is little chance for people to climb out of it even if they are willing to work.
So Good They Can't Ignore You by Cal Newport, a counter-intuitive approach to the problem of finding work you love.
The 4-Hour Workweek by Tim Ferris, something I skimmed many years ago, and wanted to revisit. The level of independence that he has achieved is enviable, but I don't completely buy the "you can do this too in N easy steps" aspect of the book. Also, as a third-worlder, I find the way he talks about hiring foreign "virtual assistants" (and paying them peanuts) incredibly entitled.

Podcast episodes:

Two episodes from This American Life about the refugee camps in Greece. (Related: A long-form NYT Magazine piece about the war in Iraq and Syria and how it got to be the mess that it currently is. Reading a synthesis like this is so much more informative than reading the latest news.)
The Buried Bodies Case from Radiolab, an ethical dilemma of epic proportions, involving the attorney-client privilege.
Playing God from Radiolab, another ethical dilemma about distributing scarce resources during an emergency.

So Good They Can't Ignore You

2016-08-15T00:00:00-07:00

I saw this book by Cal Newport on Derek Sivers's list, with a rating of 10/10 and the following review: "A MUST-READ for anyone who is not loving their work, wanting to quit their job, and follow their passion, or not sure what to do next." I picked it up from the library the very next day, and I was not disappointed. Here is a summary of the main points:

Rule #1: Don't follow your passion

The passion hypothesis: The key to occupational happiness is to first figure out what you're passionate about and to then find a job that matches this passion. This widely-held, rarely-questioned belief is wrong.
For most people, the path to work they love is messy and indirect -- it does not start with abstractly contemplating a job and then deciding to do it.
Job satisfaction has a lot to do with autonomy, competence, and connection to other people, and little to do with pre-existing "passions".
The dangers of believing the passion hypothesis: perpetual dissatisfaction with your current job, chronic job-hopping, crippling self-doubt.

Rule #2: Be so good they can't ignore you

The passion mindset focuses on what value you're getting from your job. It gets you mired in unanswerable questions like "Who am I?" and "Do I really love the work that I'm doing?" The craftsman mindset focuses instead on what value you're producing in your job. It makes you focus relentlessly on your output, and on becoming "so good they can't ignore you".
What makes for satisfying work? Creativity, impact, and control. These jobs are rare and valuable, and to get them, you have to offer something rare and valuable in exchange -- your career capital, in the form of skills you've developed. (In contrast, the passion mindset argues that all you need for a dream job is the courage to pursue it. It completely ignores skills or merit.)
Exceptions to applying the craftsman mindset:
- If your job presents few opportunities to distinguish yourself by developing relevant skills that are rare and valuable.
- If your job focuses on something that is useless or perhaps even actively bad for the world.
- If your job forces you to work with people you really dislike.
Stories of Alex Berger (TV writer) and Mike Jackson (cleantech VC) who love their work, and got there by gradually building up career capital.
The key strategy to acquiring career capital is deliberate practice. This means routinely stretching your abilities beyond your comfort level, and seeking ruthless feedback on your performance. (Deliberate practice is supposed to feel uncomfortable, in contrast to doing things you already know how to do well, which is fun.)
Musicians, athletes, and chess players engage in deliberate practice, because that's what it takes to succeed in their fields, which are winner-take-all. In contrast, knowledge workers who just show up and put in the hours tend to reach a level of basic competence at which they plateau. This is actually good news -- if you're a knowledge worker and you make a habit of deliberate practice, you'll quickly outperform your peers.

Rule #3: Turn down a promotion

What do you do once you have enough career capital? Invest it in having control over what you do and how you do it. This is a key component of a satisfying job. It increases happiness, engagement, and your sense of fulfillment.
Control trap #1: Control that's acquired without career capital is not sustainable. Example: Quitting college or quitting your job to pursue some dream occupation in which you have no experience, and in which you end up failing.
Control trap #2: Once you have acquired enough career capital to get meaningful control over your working life, you've become so valuable to your current employer that they will try to prevent you from making the change. Example: You want to downshift to 30h/week, but instead they offer you a promotion that gives you more money and more responsibility and longer hours.
In both control traps, you encounter resistance. How do you know if you're in trap #1 (and the resistance is useful) or in trap #2 (and you should ignore the resistance)? Use Derek Sivers's law of financial viability: If you're thinking of doing something that will introduce more control into your work life, do it only if you have evidence that people are willing to pay for it. Examples: Your side project has paying customers, or your employer is willing to keep you even if you drop to 30h/week. Money is a "neutral indicator of value".

Rule #4: Think small, act big

Besides control, another source of work satisfaction is building your career on a clear and compelling mission.
You need mastery in your field before you can formulate a sustainable mission. A good career mission is similar to a scientific breakthrough, in that it comes from the "adjacent possible" (see "Where Good Ideas Come From", by Steven Johnson). You need to be at the cutting edge of your field before you can see what's in the "adjacent possible".
You maximize your chances of success by making little bets -- small, achievable projects that return concrete feedback. Use these experiments to pick a direction that is most likely to yield outstanding results. (As opposed to making a big plan and betting everything on it, without testing the waters first.)
For a mission-driven project to succeed, it should be remarkable: it should compel people who see it to spread the word, and it should be launched in a venue that encourages such word-spreading (examples: academic publications, TV, open-source community).

Other notes

I found rule #4 to be less clear and convincing than the ones before.
The author uses an "hour-tally routine" where he counts the hours spent each month on deliberate practice.
The author's "little bets" are projects small enough to be completed in a month or less, and he keeps only two or three active at any time.
Summary: "Working right trumps finding the right work."

War Stories: Debugging Julia with GDB inside Docker

2016-08-15T00:00:00-07:00

One of the batch jobs at work is written in Julia, and runs for a few hours every night on a handful of beefy instances on AWS (36 cores, 64G RAM each). Recently, this job started hanging and getting killed by timeouts. Debugging this turned out to be quite an adventure. I'm recording that experience here, in case others can learn from it (or just be entertained by it).

First, because the job took some 140 core-hours, it was impractical to run the whole thing on my laptop. I tried to reproduce the problem locally by feeding the job a small subset of the data, but I could not get it to hang.

So I ssh-ed into the instance with the hung job. I saw that one of the Julia worker processes was using 100% CPU, while the rest of the system was idle. I ran strace and ltrace on the busy process, waited a few minutes, and got nothing. So it wasn't making any system calls or library calls. I started to suspect some kind of infinite loop.

Some languages have awesome tools that allow you to attach a remote debugger (Java) or a remote shell (Erlang) to a running VM, and inspect what's going on. Julia is JIT-compiled down to the metal, and does not have a real debugger, so my best bet was to attach GDB to the running process and try to see where it was stuck.

This was complicated by the fact that the Julia process was running inside a Docker container. Trying to attach gdb from outside the container didn't work, even with sudo. So I got inside the container with

docker exec -it $CONTAINER_NAME bash

and continued from there. I ran ps aux |grep julia to get the pid of the process inside the container (which is different from the pid outside the container). Then I apt-get installed gdb inside the container, and tried to attach it with gdb -p $PID. I got the following error:

ptrace: Operation not permitted.

Some googling quickly suggested adding --privileged to my docker exec command, but that did not solve the issue. More googling revealed that I also needed stop apparmor from blocking Docker's ptrace attempt, using

sudo aa-complain /etc/apparmor.d/docker

But this opened another can of worms, as I got

apparmor.common.AppArmorException: 'Syntax Error: Invalid Regex @{PROC}/{*,**^[0-9*],sys/kernel/shm*} in file: /etc/apparmor.d/docker line: 16'

Turns out this is another bug in Docker, and I had to fix the offending regex in /etc/apparmor.d/docker as follows:

- deny @{PROC}/{*,**^[0-9*],sys/kernel/shm*} wkx,
+ deny @{PROC}/{*,**^[0-9]*,sys/kernel/shm*} wkx,

After doing this, I checked the apparmor messages in /var/log/syslog to make sure that it wasn't blocking ptrace anymore. But GDB still gave me ptrace: Operation not permitted. I was stuck for a while, until a colleague helpfully reminded me to use --privileged. I learned that both fixes are necessary (the apparmor bit and the --privileged bit); neither works by itself.

So I finally managed to attach GDB to the offending process. Knowing that it was using 100% CPU for hours and hours, I used a technique known as the poor man's sampling profiler to try to figure out where the slowness was. I interrupted and continued the process several times, looking at the backtrace every time. Most backtraces looked something like this:

(gdb) bt
#0  0x00007ff4045ee59d in llvm::InstVisitor<(anonymous namespace)::SCCPSolver, void>::visit(llvm::Instruction&) [clone .constprop.345] () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#1  0x00007ff4045ef0a9 in (anonymous namespace)::SCCPSolver::Solve() [clone .constprop.343] () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#2  0x00007ff4045f4308 in (anonymous namespace)::SCCP::runOnFunction(llvm::Function&) () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#3  0x00007ff4049215bf in llvm::FPPassManager::runOnFunction(llvm::Function&) () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#4  0x00007ff4049216ae in llvm::FunctionPassManagerImpl::run(llvm::Function&) () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#5  0x00007ff4049217d3 in llvm::FunctionPassManager::run(llvm::Function&) () from /mnt/data/julia/julia-2ac304dfba/bin/../lib/julia/libjulia.so
#6  0x00007ff404047939 in to_function (li=li@entry=0x7ff204f190f0) at codegen.cpp:675
#7  0x00007ff404047aac in jl_compile (f=<optimized out>) at codegen.cpp:812
#8  0x00007ff40402001b in jl_get_specialization (f=<optimized out>, types=0x7ff202467190) at gf.c:1475
#9  0x00007ff40404e022 in emit_known_call (ff=<optimized out>, args=args@entry=0x7ff201f13010, nargs=nargs@entry=1, ctx=ctx@entry=0x7fffd70cdd50, theFptr=theFptr@entry=0x7fffd70cd610, theF=theF@entry=0x7fffd70cd618, expr=0x7ff2022b2670) at codegen.cpp:2041
#10 0x00007ff404052c99 in emit_call (expr=0x7ff2022b2670, ctx=0x7fffd70cdd50, arglen=2, args=<optimized out>) at codegen.cpp:2644
#11 emit_expr (expr=expr@entry=0x7ff2022b2670, ctx=ctx@entry=0x7fffd70cdd50, isboxed=isboxed@entry=true, valuepos=valuepos@entry=true) at codegen.cpp:3289
#12 0x00007ff4040597d9 in emit_assignment (bp=bp@entry=0x2e82dd0, r=r@entry=0x7ff2022b2670, declType=declType@entry=0x7ff202c1bf10, isVolatile=isVolatile@entry=false, used=used@entry=true, ctx=ctx@entry=0x7fffd70cdd50) at codegen.cpp:3004
#13 0x00007ff4040538f5 in emit_assignment (ctx=0x7fffd70cdd50, r=0x7ff2022b2670, l=<optimized out>) at codegen.cpp:3051
#14 emit_expr (expr=expr@entry=0x7ff2022b2650, ctx=ctx@entry=0x7fffd70cdd50, isboxed=isboxed@entry=false, valuepos=valuepos@entry=false) at codegen.cpp:3292
#15 0x00007ff404045343 in emit_function (lam=lam@entry=0x7ff2032370d0) at codegen.cpp:4835
#16 0x00007ff4040478d1 in to_function (li=li@entry=0x7ff2032370d0) at codegen.cpp:644
#17 0x00007ff404047aac in jl_compile (f=f@entry=0x7ff2022efcf0) at codegen.cpp:812
#18 0x00007ff404029329 in jl_trampoline_compile_function (sig=0x7ff202a49ac0, always_infer=0, f=0x7ff2022efcf0) at builtins.c:1015
#19 jl_trampoline (F=0x7ff2022efcf0, args=0x7fffd70ce2e8, nargs=2) at builtins.c:1030
#20 0x00007ff40401ef95 in jl_apply (nargs=2, args=0x7fffd70ce2e8, f=<optimized out>) at julia.h:1331
#21 jl_apply_generic (F=0x7ff200758db0, args=0x7fffd70ce2e8, nargs=<optimized out>) at gf.c:1708
#22 0x00007ff4050fd230 in ?? ()
#23 0x00007ff1ff634010 in ?? ()
#24 0x000000000000000c in ?? ()
#25 0x00007fffd70ce350 in ?? ()
#26 0x00000000058a9240 in ?? ()
#27 0x0000000000000000 in ?? ()

Okay, so we are deep inside the internals of Julia's compiler, the part that uses LLVM for codegen. Now what? Luckily, Julia provides some basic tools for debugging with GDB. I played around with this for a while, trying to get a stack trace of the Julia program. I wasn't able to get a full stack trace, but I did get some interesting details using variations of the following:

(gdb) f 18
#18 0x00007ff404029329 in jl_trampoline_compile_function (sig=0x7ff202a49ac0, always_infer=0, f=0x7ff2022efcf0) at builtins.c:1015
1015    in builtins.c
(gdb) p f
$28 = (jl_function_t *) 0x7ff2022efcf0
(gdb) p jl_(f)
$29 = void
(gdb) p f->env
$30 = (jl_value_t *) 0x7ff1ff634010
(gdb) p jl_(f->env)
$31 = void
(gdb) p jl_(f->linfo)
$32 = void
(gdb) p to_function(f->linfo)
$33 = (llvm::Function *) 0x6a74ff0
(gdb) p jl_dump_llvm_value(to_function(f->linfo))
$34 = void

When you do jl_(x), you get a printout of the value x in the stderr stream of the Julia process being debugged. In my case, that stream was redirected to a log file, so I got nothing in the gdb session, and had to go read the log file. I saw a bunch of unintelligible garbage, with longs strings of Float64, Float64, ..., like this:

Expr(:lambda, Array{Any, 1}[:f, :t], Array{Any, 1}[Array{Any, 1}[Array{Any, 1}[:f, Function, 0], Array{Any, 1}[:t, Tuple{Float64, Float64, Float64, Float64, ...SNIP..., Float64, Float64, Float64, Float64, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void}, 0]], Array{Any, 1}[], Array{Any, 1}[Tuple{Float64, Float64, Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}}, Tuple{Float64, Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}}, Tuple{Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}}, Tuple{Any}, Tuple{Any}, Tuple{Any}], Array{Any, 1}[]], Expr(:body,
# line 63 tuple.jl,
GenSym(5) = Expr(:call, top(:tuple), Expr(:call, f::Function, Expr(:call, Base.getfield, t::Tuple{Float64, Float64, Float64, Float64, ...SNIP..., Float64, Float64, Float64, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void}, 1)::Float64)::Any)::Tuple{Any},
GenSym(0) = Expr(:call, Base.tail, t::Tuple{Float64, Float64, Float64, Float64, ...SNIP..., Float64, Float64, Float64, Float64, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void, Void})::Tuple{Float64, Float64, Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}},
GenSym(4) = Expr(:call, top(:tuple), Expr(:call, f::Function, Expr(:call, Base.getfield, GenSym(0), 1)::Float64)::Any)::Tuple{Any},
GenSym(1) = Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, Base.argtail, GenSym(0))::Tuple{Float64, Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}},
GenSym(3) = Expr(:call, top(:tuple), Expr(:call, f::Function, Expr(:call, Base.getfield, GenSym(1), 1)::Float64)::Any)::Tuple{Any},
GenSym(2) = Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, Base.argtail, GenSym(1))::Tuple{Float64, Float64, Float64, Float64, Vararg{Union{Float64, Void}}},
Expr(:return, Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, top(:tuple), GenSym(5), Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, top(:tuple), GenSym(4), Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, top(:tuple), GenSym(3), Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, top(:tuple), Expr(:call, top(:tuple), Expr(:call, f::Function, Expr(:call, Base.getfield, GenSym(2), 1)::Float64)::Any)::Tuple{Any}, Expr(:call, Base.map, f::Function, Expr(:call, top(:_apply), Expr(:call, top(:getfield), Base, :call)::Function, Base.argtail, GenSym(2))::Tuple{Float64, Float64, Float64, Vararg{Union{Float64, Void}}})::Tuple{Any, Vararg{Any}})::Tuple{Any, Any, Vararg{Any}})::Tuple{Any, Any, Any, Vararg{Any}})::Tuple{Any, Any, Any, Any, Vararg{Any}})::Tuple{Any, Any, Any, Any, Any, Vararg{Any}})::Any)::Tuple{Any, Any, Any, Any, Any, Vararg{Any}})::Any

I looked at line 63 of tuple.jl in the version of Julia that I was running, but that was utterly unenlightening. After a few hours spend on this, I didn't seem to be getting anywhere. Disgusted, I went home and slept.

The next day, I was thinking about the Vararg{Union{Float64, Void}} bits seen in the dump above, and I had two strokes of insight. First: Vararg meant that there was a function with a variable number of arguments, using "splat" syntax like f(arg1, arg2, more_args...). The program was about 1500 lines, and there were about ten places using the splat syntax. This narrowed down the search space for the bug considerably.

Second: Union{Float64, Void} meant that Julia was dealing with a value that could be either a float or a nothing. That was weird. I couldn't think of where that might happen, but then I realized that we were parsing a bunch of float arrays from JSON. Could a null somehow sneak in among those floats? A quick scan of our input dataset, using jq and grep, confirmed this hypothesis: the arrays were supposed to be all floats, but contained the occasional null. (This was the result of serializing NaN values into JSON, which in turn was due to a freshly introduced bug in another system.)

Putting these two insights together, I was able to pinpoint the location in our Julia code that was the cause of the apparent infinite loop. It was an array concatenation using splat syntax, like result = [x..., y...]. When x and y only contained floats, this was not an issue. But when one of them contained an occasional nothing, Julia's type inference for the resulting array went haywire, and took time exponential in the length of the arrays. Here's a minimal example to illustrate this exponential growth in running time (in Julia 0.4.6):

julia> for n in [100, 150, 200, 250, 300, 350, 400]
           x = zeros(Float64, n)
           y = [0.0, nothing]
           print("n=$n: ")
           @time result = [x..., y...]
       end
n=100:   0.711594 seconds (208.66 k allocations: 12.082 MB, 3.53% gc time)
n=150:   1.409423 seconds (88.71 k allocations: 4.886 MB)
n=200:   2.961411 seconds (100.57 k allocations: 5.549 MB)
n=250:   7.311390 seconds (126.25 k allocations: 6.930 MB, 0.07% gc time)
n=300:  16.698541 seconds (153.91 k allocations: 8.400 MB)
n=350:  25.183418 seconds (183.35 k allocations: 9.966 MB, 0.02% gc time)
n=400:  39.814280 seconds (214.75 k allocations: 11.587 MB)

So it wasn't an infinite loop at all, just an incredibly slow operation. (This also explains why I did not observe any problem when running the program locally on a subset of the data.) The workaround was to use result = Any[x..., y...] or result = vcat(x, y), thus constraining the type of the resulting array, and avoiding this worst-case inference behavior. I actually went with result = Float64[x..., y...], since I wanted the program to fail hard if there were any nothings in the arrays.

I bundled up the minimal example and the workarounds and submitted a bug report in Julia, feeling a mixture of triumph and futility at being able to track down this nasty bug, just so a few bits of the Internet will continue to work for a few more days.

June and July Reads

2016-08-05T00:00:00-07:00

These past two months I've been pretty busy, as I took my parents on their first trip to the US, and also released the new mobile-friendly version of Tinyblu, which has been in the works for a long time. So I haven't had much time for reading. But there are still a few things worth mentioning:

The Fix Is In, a This American Life episode about collusion by companies to suspend competition so that they can keep prices up. It's fascinating because they actually got these guys on tape talking about fixing prices!
The Problem with the Solution, an Invisibilia episode about the American drive to find solutions to problems, and what that means for mental illness in particular. I'm so excited that Invisibilia season 2 is now live!
Flash Boys, Michael Lewis's take on high-frequency trading.

I'm also currently on a binge of leafing through the Moon Living Abroad series of travel guides ;) It helps keep the wanderlust at bay, at least somewhat.

May Reads

2016-06-06T00:00:00-07:00

How Technology Hijacks People's Minds shows some great examples of psychological tricks used to guide people's attention and their choices. The incentive of any free, ad-supported website or app is to get you to spend more time on it, which is probably not your goal.
The Case for Reparations shattered any illusions I might have had that the US had reached some kind of racial harmony.
This American Life episode The Middle of Nowhere got me to read about Nauru, the Tampa affair, the Pacific solution, and Australia's harsh approach to dealing with refugees.
Love, Dishonor, Marry, Die, Cherish, Perish was a fun, short novel written in rhyming couplets!

April Reads

2016-05-09T00:00:00-07:00

Books:

Physics for Future Presidents is full of fascinating facts and busted myths about energy, nuclear power, nuclear bombs, global warming, and other topics that are hotly debated in the media.
The Big Short tells the story of the financial crisis from the point of view of the few people who realized what was going on, and made a killing out of it.

Cool stuff online:

A nuanced TechCrunch article about the SF housing affordability crisis and the history that led up to it.
A bunch of articles about homelessness: how much SF spends with little accountability, how Utah did or did not solve its homelessness problem, and a character story called Million Dollar Murray.
A TED talk by Nick Hanauer, intriguing because you don't often hear rich people wanting to curb income inequality.
The 1954 Guatemalan coup d'état, orchestrated by the US government with the involvement of the United Fruit Company -- a fascinating piece of history that I hadn't known before, and a revealing example of US foreign policy.

February and March Reads

2016-03-31T00:00:00-07:00

Books:

The Checklist Manifesto by Atul Gawande -- a short and engaging read about using checklists to reduce human errors in aviation, construction, and now medicine. We could probably use a lot of these lessons in software development and deployment.
Ishmael by Daniel Quinn -- one of those books that I stayed up late to finish. It exposes the unquestioned anthropocentric assumptions at the core of our civilization, and asks whether by taking our destinies into our own hands (as opposed to letting "the gods" look after us, as in hunter-gatherer societies), we have also chosen our destruction. This got me reading about projected population growth in the coming century. Some time ago I ran across this paper by Lutz et al. (2001) and felt reassured, but now I looked at the UN projections from 2015, and they paint a different picture: there is only a 23% chance that global population will stabilize or begin to fall by 2100. (On the other hand, so much could change in the coming 100 years, that any long-range forecast has to be taken with a proportionately large grain of salt.)

Podcasts:

60 Words by Radiolab dives deep into the Authorization for Use of Military Force -- the 60 words that became the legal foundation for the "war on terror" after 9/11.
Poop Train by Radiolab traces what happens to New York City's sewage, and ends up in a very interesting place.
Planet Money reveals how easy it is to set up a shell company abroad.

Cool stuff online:

A report by the Silicon Valley Competitiveness and Innovation Project has some numbers that caught my eye: 58% of STEM professionals in Silicon Valley are foreign-born, and Americans are moving out of this area faster than they are moving in.
Mr. Money Mustache writes about early retirement (in detail) and corrects my misconception that it's all about extreme frugality.
A Slate article about gentrification in Oakland shows the larger historic arc of population movement, without the usual scapegoating narrative.
Tim Urban decides he wants to live forever, and makes a good argument for cryonics.

December and January Reads

2016-01-31T00:00:00-08:00

Essays:

This Is Water by David Foster Wallace -- a thought-provoking commencement speech about empathy and the meaning of education.
Normalization of Deviance in Software by Dan Luu -- a great essay about how broken practices become the norm, instead of getting fixed.

Books:

The Signal and the Noise by Nate Silver -- a captivating tour of what we can and can't accurately predict.
The Last Girlfriend on Earth by Simon Rich -- so funny that I started recommending it to friends.

Podcasts:

Birthstory by Radiolab -- a fascinating and disturbing look at the market for surrogate pregnancies.
Brown box by Radiolab -- a look inside an online retailer's fulfillment center, and what it takes to give you next-day delivery.

The Signal and the Noise

2016-01-30T00:00:00-08:00

Nate Silver's book has been a lot of fun to read. There is too much good information in it for me to attempt a comprehensive summary, so instead, here are a few points that I found especially interesting:

There is a common perception that houses are good investments, because they increase in value over time. But US home prices have been pretty constant over the long run, according to the Case-Shiller index.
Weather predictions like "40% chance of rain" are obtained by running a deterministic simulation multiple times, with slightly different initial parameters. It's not a stochastic simulation, like I thought. Increasing computational power has made weather and hurricane forecasts much better over time, but still, the weather is such a chaotic system that forecasts 8-9 days into the future are no better than historical climate averages.
The efficient-market hypothesis explains why, for practical purposes, you probably can't beat the stock market as an individual investor. But if you are a professional trader, there are cases when it's rational to buy, even when the chance of an upcoming crash is high.
The author made some $400K in the "online poker bubble" of 2003-2006. But making money in poker relies on having lots of "fish" in the game -- bad players who lose money.
Statistical significance as measured by p-values is a broken way to determine the truth of a hypothesis. Here is a cool demonstration of p-hacking -- slicing and dicing your data in order to get p < 0.05 so that you can get published.
Most political and economic forecasts are garbage, but prediction markets offer an interesting alternative. Making bets based on your beliefs (in the probabilistic sense of "belief") is the essence of being Bayesian.
Earthquakes and terrorist attacks obey a power law distribution. Each individual event is mostly unpredictable, but we can make statements about the frequency of events of a given magnitude. The plots in the book suggest that a 9/11-scale terrorist attack (killing 3,000 people) would happen once every 40 years, and that larger-scale attacks are a clear possibility.

Generating Two-Factor Authentication Codes on Linux

2016-01-16T00:00:00-08:00

I recently almost-broke my phone. If I had fully broken it, I would be locked out of most of my accounts, because I use Google Authenticator for two-factor authentication on every account that supports this option. Here is a quick note about how to generate authentication codes on your Linux machine, in the same way that Google Authenticator does on your phone.

First, you need to find out the TOTP secret for your account. This secret is displayed in plain text when you set up two-factor authentication, as highlighted in green in the image below.

For sites that only display a QR code, you can take a screenshot and parse the QR code using the zbarimg command (provided by the zbar-tools package). You don't even need to crop the image beforehand.

$ zbarimg screenshot.png
QR-Code:otpauth://totp/<ACCOUNT>?secret=<SECRET>&issuer=<ISSUER>

Treat the TOTP secret like a password. Save it to an encrypted file, or use a password manager. Don't store it somewhere in plain text, and definitely don't email it to yourself.

Once you have your secret, you can generate authentication codes using the oathtool command (provided by the oathtool package):

oathtool --base32 --totp "<SECRET>"

You can verify that this generates the same codes as Google Authenticator on your phone.

Writing a Novel in 30 Days

2016-01-03T00:00:00-08:00

NaNoWriMo is a fun, crazy creative writing challenge: write a 50,000-word novel during the month of November. I've known about it since at least 2011, but I always convinced myself that I was too busy to participate. Until now! In 2015 I finally rolled up my sleeves and wrote my 50K words. In this post I'm going to talk about how it went and what I've learned.

How long is 50K words? It's a short novel. To compare, the first Harry Potter book is 77K words, and George Orwell's 1984 is 89K words. 50K words is only two times longer than the Wait But Why post about Tesla, and it doesn't even include Tim Urban's awesome illustrations. On the other hand, it's definitely the longest thing I've ever written: about 8 times longer than my longest blog post, and about 22 times longer than my longest email.

Writing 50K words sounds intimidating, but as I learned from NaNoWriMo, that is the wrong way to think about it. If you think about it as writing 1,667 words per day, it becomes much more manageable. In fact, I found that my progress was more predictable than on software engineering projects, because the number of words provides an unambiguous progress bar that doesn't exist for software.

I did a bit of preparation before November, but not much. I started by reading some of the great resources that NaNoWriMo provides. I made a character sheet and a half, although when I started writing, my characters kind of ran away from me and I had to get to know them all over again. I had a basic idea for the plot, but I ended up changing it at least five times during the month of writing, in places where it seemed too predictable. I also told a few friends that I was doing NaNoWriMo, so that they would keep me accountable and make it less easy to bail out.

Then November came and the roller-coaster got rolling. In the first half of the month I was still waiting for my work authorization to clear, so I was at home working on side projects. I took advantage of the free time and aimed for 2K words per day, instead of the customary 1,667. It didn't even seem that difficult. But then I started at my new job on November 16, and holding on to my daily word quota became a lot harder. There were days when my routine was reduced to: wake up, commute, work, commute, eat dinner, write, go to sleep, repeat. I was tired and sleep-deprived and excusing myself from all things social, which is definitely not sustainable for more than a few weeks. I was also worried about messing up right before the finish line, and reaching December 1 with only a few thousand words left to write. But the buffer I gave myself by writing 2K words per day in the first half of the month helped a lot, and I finally reached 50K during Thanksgiving weekend.

NaNoWriMo encourages people to turn off their inner critics, write a first draft for speed instead of quality, and worry about editing later. This is the opposite of my usual perfectionistic approach, where I edit as I write, and the first draft is the only draft. So this has been an interesting exercise in writing quickly and resisting the temptation to edit.

Before November, I ran a "speed test" to estimate how big of a time commitment NaNoWriMo was going to be. I typed 2,000 words in stream-of-consciousness style, writing whatever came into my head, and not worrying about content. I achieved a speed of 29.5 words per minute (WPM) this way. During NaNoWriMo itself my writing speed was much lower: from 15.3 WPM on the fastest day to 5.2 WPM on the slowest day, averaging 9.7 WPM. The chart below shows the stats for each day, where the top bars indicate the number of words, the bottom bars indicate the time spent writing, and the circles show words per minute. This chart does not include a few additional hours that I spent doing character sheets, outlining, and obsessing over the plot.

I noticed a few factors that influenced my writing speed. On some days I had a detailed phase outline of what I was going to write, so it went pretty quickly. On other days I had no outline, or only a loose one, and the writing was slower. But subjectively, the most original parts of the story came to me when I wasn't writing from an outline. Towards the end of the month, I ran out of plot and still had a few thousand words left to write, so I had to invent a bit of filler, which also slowed things down.

Unsurprisingly, my success at shutting up my inner critic was also correlated with writing speed. When I thought of the story as an experiment that no one was going to read, I was able to write pretty quickly. But the minute I started to think it might be a story worth telling, my writing speed went way down. In the end, I'm still more comfortable with my original "write slowly, edit as you go" style than with NaNoWriMo's "write fast, edit later" approach.

My story is about two characters, one male and one female, both biologically of college age, but emotionally a little younger. I decided pretty early on that I wanted to write half of the story from a female first-person point of view, which was a lot of fun, but I don't know how credible. There is a lot of angst, a lot of lust, and a bit of murder mystery going on. The most honest way to describe it is as a young-adult wish fulfillment novel.

So now what? I'll probably clean the text up a bit sometime this year, and post it on this blog for the occasional Internet wanderer to thumb through. (The first draft is too rough to post right now, but I've given it to a few friends for feedback, so if you're interested, email me.) Then I'll go back to writing shorter pieces and focusing on content rather than length. I'd love to establish a daily writing habit. The most emotional scenes to write for NaNoWriMo were actually about parents and children and growing up and leaving home, which took me completely off guard. Maybe I should write some more about that.

November Reads

2015-11-30T00:00:00-08:00

Podcasts:

Invisibilia was love at first episode, followed by an uncontrollable listening binge, followed by impatience for season 2 to start. My favorite stories were that of a man emerging out of locked-in syndrome and recounting his experience, and that of a blind man learning to echo-locate, but there are more that are just as good.
Radiolab's story about the oppression of the Mau Mau in colonial Kenya. How many more stories like this one are hidden away in archives that are not open to the public?
Planet Money's episode on the unexpected forces that kept the cost of a Coca Cola can at 5 cents for over 70 years.

Cool stuff online:

The Cougar Ace story, about rescuing a huge container ship that tipped on its side and almost sank. The text, the photos, the men involved -- everything feels like an action movie, even though this is a true story.
Tim Urban's finale on the topic of Elon Musk on Wait But Why. Long and funny and inspirational. I wish I could remember more often to treat life as one long game of Grand Theft Auto.

I also read Alice Munro's short story collection The Love of a Good Woman. This book sneaked up on me. The first few stories were not that interesting, but then they kept getting more and more intense, until the last two, which were my favorites (Before the Change and My Mother's Dream).

October Reads

2015-10-31T00:00:00-07:00

Books:

Wild, a memoir by Cheryl Strayed about hiking the Pacific Crest Trail on her own. This book has a strong female voice that is deeply personal and unapologetically honest. As a guy, I found some passages both thought provoking and enlightening.
The Orphan Master's Son, a novel by Adam Johnson about life in North Korea. This was more disturbing than I thought, and it made me appreciate all the things we take for granted in the West.

Podcasts:

After the Flood, a This American Life episode about the aftermath of Katrina. All you need to know about rich / poor and white / black relations in America.
The M&M Anomaly, where Planet Money asks why peanut butter M&M packs weigh slightly less than standard M&M packs. There's a whole fascinating and secretive world behind that question.

Cool stuff online:

The strange story of Anna Stubblefield and facilitated communication, a controversial technique for interacting with severely disabled people.
Patrick McKenzie's blog, where he chronicles his career as an independent software vendor and consultant, making his income this way and living without a day job. He is remarkably open about his thoughts and strategies, and even the financials of his projects. I especially enjoyed his year-in-review posts from 2006 onwards, found at the bottom of this page.

Stumbling on Happiness

2015-10-19T00:00:00-07:00

I've read Daniel Gilbert's Stumbling on Happiness back in 2008, just before going to college, and I wanted to read it again now that I'm navigating the transition out of grad school and into the workforce. This book is not a manual for how to be happy -- instead, it presents the reasons why happiness is so elusive, and why we work so hard towards things that ultimately don't make us as happy as we thought they would. The book is engaging, funny, accessible, well organized, and full of citations -- everything that a good popular science book should be. If you like learning about human nature, I highly recommend it. In this post I will summarize the book's main points. The author also has a popular TED talk about synthesizing happiness.

Part I, Prospection, asks why we look into the future in the first place. Well, we like to imagine ourselves being successful and happy in the future. We also like to anticipate potential negative outcomes, so that we can prepare for them. We want to feel in control of our future experiences, we want to matter, to make things happen. (Losing this feeling of control over one's life is a hallmark of depression.) But why is controlling our future so important to us? Here Gilbert provides two answers: First, we want to be in control for its own sake, because exercising control is rewarding and it makes us feel good. Second, we want to be in control because some futures are better than others, and we want to steer our lives towards the futures that will bring us the most happiness. The second answer sounds like common sense, but it is in fact wrong, because the future is fundamentally different from how we imagine it. This is the central argument that the rest of the book lays out.

Part II, Subjectivity, tackles the problems of defining and measuring happiness. It's hard to pin down a subjective experience like emotional happiness, so Gilbert settles for "the you-know-what-I-mean feeling". This is different from moral happiness ("I should be happy because I live according to moral standards") and from judgmental happiness ("I am happy that Larry Lessig is running for president").

Now that we know what we mean by emotional happiness, all that remains is to measure it. For reasons that philosophers like to argue about (language squishing, experience stretching), comparing the happiness of two different people is a difficult proposition. Even comparing your current happiness to your past happiness is tricky, because your memory of the past is colored by your present point of view. Gilbert concludes that the best happiness meter is the honest, real-time report of an attentive person. It's not perfect, but it's the best we have. And by measuring the happiness of many different people, we allow the law of large numbers to cancel out any individual calibration errors.

Armed with a way to quantify and compare happiness, the book goes on to illuminate three types of foresight errors -- ways in which our imagination misleads us about how the future is going to be.

Part III, Realism, exposes imagination's first flaw: the tendency to fill things in and leave things out. This happens effortlessly and without our awareness, so we tend to assume that our imagined scenario accurately reflects reality. For example, when we think of George Eastman's successful career as an inventor, we fill in that he must've been quite satisfied, and so we're baffled to learn that he shot himself. Our imagination leaves out the health problems that he had later in life, and which might explain his decision.

Similarly, when we imagine a future event, we sketch out a few salient features, and leave out many others. Because of this incomplete picture, we can't accurately predict how happy some particular future will make us. We might imagine the fat paycheck but not the stressful work, and thus overestimate our future happiness at a particular job. Or we might imagine the arduous hike but not the glorious sunset, and thus underestimate how happy we'll be after a day outdoors.

Gilbert draws parallels between the filling-in and leaving-out of imagination and similar flaws in memory and perception. When we remember something, it feels like we're replaying a movie, but in fact, we're reconstructing the movie from a compressed representation, and this process of reconstruction can lead to memory errors. Similarly, our visual system "fills in" the blind spot on our retina, making us susceptible to visual illusions.

Part IV, Presentism, examines imagination's second flaw: the fact that when we imagine the future, we borrow liberally from the present. This is why when you're satiated, it's hard to imagine feeling hungry, and when you're depressed, it's hard to imagine ever enjoying life again. (It's also why old sci-fi books seem hopelessly anachronistic.) The present colors not only imaginings of the future, but also recollections of the past, especially recalling how you felt at a particular time.

The same areas of the brain that are used for processing visual stimuli, auditory stimuli, and emotions are active when remembering or imagining visual scenes, sounds, and emotions. The brain has a reality-first policy, which is why it helps to close your eyes when you're trying to imagine a penguin. And we're pretty good at distinguishing between a visual or auditory image that comes from the environment, versus one that comes from our imagination. But not so with emotions -- what's the equivalent of closing your eyes to shut off satiety when you try to imagine hunger? So we mistakenly conclude that we'll feel tomorrow as we feel today.

Other subtle forms of presentism include:

Failing to account for habituation (the fact that the first kiss feels much better than the one hundredth). Two workarounds: add variety, or increase the time interval between repetitions.
Anchoring. Since mental images are atemporal, when we imagine how something will feel in the future, we actually imagine how it would feel right now, and then make a correction for the fact that it will happen later. We often don't correct far enough.
Making a comparison in the present and assuming that we'll be making the same comparison in the future. This includes all the favorite tricks of behavioral economists, such as the fact that a $X loss seems more powerful than a $X gain, even though they are equivalent in expectation.

Part V, Rationalization, explores imagination's third flaw: the failure to recognize that we'll feel differently about something once it's already happened. When we contemplate a negative experience, we tend to think that it will affect us intensely and for a long time. But people who actually go through a traumatic experience adapt pretty quickly, and soon say that they feel just as happy as everyone else, and even enhanced by the experience. We are less fragile and more resilient than we think, thanks in part to our psychological defense system.

This defense system is made possible by the ambiguities pervading the world around us, and it is especially good at making us feel better about intensely negative and inescapable situations. It also has other effects that arguably make us less rational:

We surround ourselves with people who agree with us.
We seek evidence confirming our beliefs, and when we find it, we accept it uncritically.
When we encounter evidence against our beliefs, we examine it more stringently, and often find reasons to dismiss it.
We tend to evaluate a choice (which job to take, which appliance to buy, etc.) more positively after we've already made it.

Part VI, Corrigibility, asks why we never get better at predicting our emotional futures. Usually, when we're bad at something, we get better in two ways: by learning first-hand from our own experience and mistakes, and by learning second-hand from the wisdom of other people around us. So why does it seem like we never learn, when it comes to predicting our own future happiness?

Experience doesn't help much because it relies on accurately remembering how we felt in the past, which is not something our memory systems do very well. We tend to remember unusual occurrences -- the best of times and the worst of times -- rather than the common, ordinary times. We place more weight on how something ends, which is why a bad ending can ruin an otherwise good movie. And we rationalize how we must've felt in the past ("It was Monday morning, so I must've felt grumpy") , instead of actually remembering.

Cultural wisdom doesn't help much either, because memes about happiness spread not on the basis of goodness or truth, but on the basis of how good the memes are at spreading themselves. For example, there is evidence that once they've reached the middle class, people don't get happier with more money. And yet, the belief that money brings happiness is widespread, because it keeps the economy going, which contributes to a stable society, which in turn helps this belief spread. As another example, there is evidence that married couples are happiest before they have children and after the children leave home. And yet, the belief that children bring happiness is common, because any society where this weren't the case would peacefully end in one generation.

So if we can't learn from our own experience or from the wisdom of others, what can we do? Are we doomed to always be poor at predicting what will make us happy? Gilbert proposes a simple solution, but one that most people find unappealing. The solution is this: When you consider how experience X will make you feel, stop relying on your imagination. Instead, find a surrogate -- someone having that experience right now -- and ask them how they feel. This is deeply unsatisfying because we like to feel special, and we don't think that someone else's experience will transfer to us. But given the limits of our imagination, this might be the most accurate way to predict how happy something will make us feel.

Trip to the Grand Canyon

2015-10-17T00:00:00-07:00

Red cliffs, epic dams, and photographers' heaven -- Antelope Canyon, above.

(more pictures)

September Reads

2015-09-30T00:00:00-07:00

I've been traveling this month, and haven't done much reading. I did read Murakami's What I Talk About When I Talk About Running though, and I was surprised to learn that he started as a writer pretty late (and on a whim). He also ran 23+ marathons.

I started listening to podcasts instead of music when I cook, and I immediately found two favorite episodes from This American Life:

The Giant Pool of Money explains the 2007-2008 financial crisis through the voices of several people who were involved in some capacity or another. I didn't really have a big-picture understanding of what happened until I listened to this episode.
129 Cars offers a glimpse at the inner workings of a car dealership, where things get way more dramatic than I anticipated. Who knew they sometimes sold new cars at a loss, just to meet their sales quota.

Also, time capsules! There is something fascinating about sending messages into the future. Especially if that future is so distant that you can't count on current languages and measurements units to still be around. The Westinghouse time capsules buried in New York are supposed to be opened in 6939. The KEO satellite (not yet launched) is supposed to carry messages to people 50,000 years from now. And the LAGEOS plaque has an even longer timespan: 8.4 million years before the satellite carrying it re-enters the atmosphere.

August Reads

2015-08-31T00:00:00-07:00

I'm starting a series of posts collecting the best articles / books / etc. that I've read each month. Here we go, the coolest things I've read this August!

Two black-swan events that we are completely unprepared for, because our history doesn't stretch back long enough:

An earthquake in 1700 that would destroy Portland, Seattle, and Vancouver if it were to happen again.
A volcanic eruption in 1815 that caused a volcanic winter -- blizzards in New York in June, dead crops and famine worldwide.

Other random things:

How easy it is to fabricate scientific findings, and how little scrutiny there is in the worlds of publishing and journalism.
The "door to hell" in Turkmenistan is a place I can't believe exists.
Tim Urban's epic SpaceX post on Wait But Why.
Don't Call Yourself A Programmer, And Other Career Advice.

I also finally read The Hitchhiker's Guide to the Galaxy, which was way more hilarious than I expected! Fun fact: the book was published in 1979, and one of the computers in the book is called the Googleplex. Google was started in 1998, yet its official naming story does not mention the book. It makes me wonder.

August Reads

2015-08-31T00:00:00-07:00

I'm starting a series of posts collecting the best articles / books / etc. that I've read each month. Here we go, the coolest things I've read this August!

Two black-swan events that we are completely unprepared for, because our history doesn't stretch back long enough:

An earthquake in 1700 that would destroy Portland, Seattle, and Vancouver if it were to happen again.
A volcanic eruption in 1815 that caused a volcanic winter -- blizzards in New York in June, dead crops and famine worldwide.

Other random things:

How easy it is to fabricate scientific findings, and how little scrutiny there is in the worlds of publishing and journalism.
The "door to hell" in Turkmenistan is a place I can't believe exists.
Tim Urban's epic SpaceX post on Wait But Why.
Don't Call Yourself A Programmer, And Other Career Advice.

Road Trip in Ecuador

2015-06-04T00:00:00-07:00

(This trip happened at the end of March, but it took a long time to write the blog post.)

After last year's road trip through California, Raunak and I got more ambitious and went to Ecuador this spring break. We broke a car, drove through a cloud, explored the rainforest, and found the most beautiful beach in the world. Keep reading for the full details, see some pictures here, and check out Raunak's post for all the birds we've seen on our trip.

Day 0: An inauspicious beginning

After a few hours in the company of an exceedingly cuddlesome couple seated next to me, and a flight attendant who looked like Hitler, I landed in Quito around midnight. Raunak had arrived an hour earlier, and had to wait for me while I slogged my way through border control, and was finally wished bienvenido.

At the car rental office, we had our first taste of interacting with the locals using our non-existent Spanish. (Hint: It takes a good dictionary and lots of gesticulation.) We signed the paperwork and were on our way. But we didn't even get out of the parking lot, because we couldn't figure out how to put the car in reverse. We pulled the gear shift every which way, until it got kind of slack, and it wouldn't even switch between the forward gears anymore. That is how you break a car in ten minutes. The rental guy gave us another one, and showed us how to reverse -- you had to pull up a ring on the gear stick.

Because the airport had recently moved to a new location, the hotel we got for our first night in Ecuador was quite far away. The nighttime and the heavy rain made it difficult to orient ourselves, and we took a few wrong turns that put us on divided highways in the wrong direction. After a huge detour, we finally got to our hotel around 2 AM. We were a bit dismayed with the weather forecast, which showed solid rain for ten days. But our luck was about to turn.

Day 1: Mindo

The hotel's complimentary breakfast was surprisingly delicious -- croissants and fruits and queso fresco -- nothing like the stale bagel and cream cheese you get in the US. We went to buy SIM cards, and then hurried out of town. Gas was only $1.50 per gallon -- half the price in California at the time. (Petroleum is Ecuador's largest export.) We stopped at a supermercado to get some fruits -- bananas were $1 per dozen. (Bananas are Ecuador's second largest export.) The "supermarket" was little more than a farmer's market under a roof, which reminded me of the bazaars back home.

Our first destination was Mindo -- a town that was recommended to us for birdwatching. The road was beautiful, with a wall of green on each side. Traffic was slow, but we made it fun by reading off license plates to practice our Spanish numerals.

Mindo turned out to be quite touristy, with places for rafting and ziplining, and English-speaking guides that charged American prices. We spent the afternoon hiking up a hill, accompanied by three friendly canine guardians with too much time on their paws. None of the vegetation looked familiar, and even the ordinary grass on the ground was different from the one I was used to. We saw a few species of birds, a red centipede, and an agouti scurrying across the trail. Mosquitoes were miraculously absent, perhaps because of the elevation.

When we got back to town, it was already dark. (Ecuador has long nights, with the sun rising at 6 AM and setting at 6 PM all year round.) We hired a birding guide for the next day, and, being cheap, slept in our car in front of his house.

Day 2: Papallacta

We slept pretty well, despite the rain outside and the insomniac rooster crowing at 1:20 AM. We got up before sunrise, and our guide Alex directed us to a cock-of-the-rock lek -- a place where male birds came together to show off in the hopes of attracting a female. We gained some trust in our Chevy Aveo, as it braved the unpaved roads without complaint. For the rest of the morning we went to more birdwatching spots, seeing a quetzal with bright red and green feathers, and an adorable potoo sleeping on a stump.

On the drive back from Mindo, we learned how things worked on Ecuadorian highways. A bird's-eye view would show a slow-moving truck, with a long chain of smaller vehicles stuck behind it. After that, a refreshingly empty stretch of road, until the next truck, with its own procession of frustrated tailgaters. We quickly decided that driving in Ecuador was a game, where you got points for overtaking such trucks. At first we were quite cautious, following the dashed and solid lines like we would do in the US. This meant losing a lot of points. Later on in our trip, as we learned more about Ecuadorian driving, we would get much better at this game.

Our next destination was the Antisana volcano. Despite being an ecological reserve, Antisana is surrounded by private property, which requires written permission to pass through. We spent a few hours in Quito trying to reach a certain señor José Delgado, but we could not find his current phone number, so we gave up on Antisana and moved on. (This lack of transparency and online information about access to national parks proved to be a recurring theme on our trip. It's very profitable for the tour agencies that take care of everything for you, but it sucks for the independent tourist.)

On our way east from Quito we ended up on a toll road. We didn't know what the toll was, so we handed the attendant a $20 bill, thinking it should be enough. She handed us back $19.40. (Ecuador does not have its own currency, and uses the US dollar instead. Currency substitution is a fascinating topic that was completely off my radar before this trip.)

We drove to Papallacta, a hot-springs town with access to the Cayambe Coca Ecological Reserve. For dinner we stopped at a local family-run restaurant, where the owner insisted that we take a picture and post it on Facebook to advertise her establishment. We were planning to sleep in the car again, but then she made us an offer we couldn't refuse: a room upstairs for $20, with a hot tub in the backyard. We spent the evening in that tub, fantasizing about retiring to a place like this, where the faucets have only one lever, and the water that comes out is hot.

Day 3: Cayambe Coca

In the morning we drove up the long dirt road to the Cayambe Coca entrance, and then hiked towards one of the nearby lakes. The dry, shrubby vegetation reminded me of California. But the ground under our feet was soft and muddy, and there was moss all over the place. We also found a strange plant that grew in patches, and some of them looked like perfect pillows:

We were at an elevation of about 3600 meters, and I was surprised at how weak I felt. Even a slight uphill made my heart beat heavily, and my lungs struggle for air, with an unpleasant panicky feeling that forced me to slow down. (I never thought about it before, but I've lived all my life in cities less than 100 meters above sea level.)

The clouds cleared on the way back, and we got a magnificent view of Antisana in the distance:

We decided to camp in Cayambe Coca the following night. Going to nearby Baeza for food and gas, we learned more about Ecuadorian driving: A big bus kept cutting into the wrong side of the road to avoid slowing down at the curves. Back at Cayambe Coca, we did another short trail to a waterfall:

The area near the waterfall looked like some kind of alien landscape, where every inch of every branch was covered with moss:

We picked a camping spot near one of the lakes, and set up our tent. It had drizzled earlier in the day, and we were expecting heavier rain at night. That's when we discovered that we didn't have a tarp to cover the tent. We figured we would use Raunak's poncho in place of a tarp, but securing the poncho against the wind took a bit of ingenuity. We put apples inside Raunak's gloves, which had velcro straps, and attached the gloves to two corners of the poncho. In the remaining two corners we put mangoes, and tied them up with a pair of cables that I happened to have in my backpack. We felt very accomplished.

Day 4: Driving to Coca

That night my stomach decided to stop working, and I had to go outside and do a manual restart. Luckily I felt better the next morning. After all our worrying about rain, it never actually more than drizzled. It was the cold that we should've worried about -- when we awoke, we found ice on the inner seams of our tent.

We hiked to another lake, reaching an elevation of 3950 meters. The clear sky changed the scenery completely. Here is what it looked like yesterday:

And here is what it looked like this morning:

After the hike, we got on the road towards Coca, our next destination. From there we hoped to enter the Yasuni national park, which is part of the Amazon rainforest. I drove all the way there -- my first time driving a manual car! It wasn't as complicated as I had thought, except for starting the car... more on that later. At first we were on windy mountain roads, and then we got to the flat, straight roads on the east side, where even 100 km/h felt slow. We saw the vegetation change several times as we descended from nearly 4000 meters to about 400. My eyes couldn't get enough of that verdant abundance; there were even people with machetes chopping off the greenery to prevent it from invading the road. (Ecuador has impressive first-world highways, but all the other roads are third-world like you'd expect.)

Once in Coca, we thought we'd take some small roads out of town and end up in the rainforest. Instead, we found that all the green stuff on Google Maps was just boring countryside. We returned to Coca, got a hotel room for the night, and went out looking for a map. We found lots of people selling guided tours, but no information about how to get to Yasuni ourselves. (Some people have pulled it off, but it's not nearly as easy as going to a national park in the US. I got the impression that most tourists went to one of the uber-expensive "eco lodges", and we did not want to consume a pre-packaged experience like that.) After some deliberation, we decided to spring for a three-day tour with a local guide, who was willing to take just the two of us, instead of a larger group.

Day 5: Monkey Island

We met our guide, Sandro, at the ferry terminal in the morning. Our boat was long and narrow, made of wood, and it had a sun shade. The motor fed directly from a big barrel of oil sitting behind us. As we headed east on Rio Napo, Sandro was at the back, steering, while we kept our eyes peeled for birds and other creatures.

Our first stop was the "monkey island", a sort of halfway house for ex-captive monkeys being reintroduced to the wild. We met the family living on the island and running the project. There were a bunch of wooden huts with thatched roofs, big plastic barrels to collect rainwater, and chickens running freely all over the place. We put on rubber boots and went for a tour of the island. First we noticed the insects: The cicadas making an incredible racket, the two-inch mosquitoes shimmering in the sun, and the large colorful butterflies posing for a picture:

Sandro called our attention to the walking palm trees around us -- something straight out of Lord of the Rings. At first the rubber boots served us well, but then we reached a part of the trail where the water was up to our thighs. We plowed through, letting our boots fill with water, and then continued our hike with comical sloshing sounds.

Despite the relatively large number of monkeys, the island was not at all like a zoo. We went a long way without seeing any of them, until finally we spotted a pack of woolly monkeys jumping from tree to tree. Even then we needed binoculars, and the view was obscured by the very dense vegetation. This was my first time seeing monkeys in the wild.

Food was part of the tour, and when we got back to the wooden huts, Sandro made a delicious tuna salad. We learned that he was part of the Kichwa tribe, and that his wife was Russian. Sandro spoke a little English, and with the help of a dictionary, we spoke a little Spanish, so we managed to communicate surprisingly well. After lunch, we got on the boat and continued to travel east.

We briefly stopped by Sandro's cousin's house north of the river, but other than that, we were on the water for a few uninterrupted hours. Sometimes Sandro would steer the boat near the shore and slow down, and we'd see all kinds of birds, including the bizarre-looking hoatzin. The landscape was strikingly flat: there were no hills or mountains in any direction, and even the bottoms of the clouds looked like someone had taken them and squashed them against a pane of glass.

We reached an area where the water was very still, and Sandro showed us how fishing was done on the Rio Napo. You take a bamboo stick, and fix it into the shallow bottom. Then you tie a long and narrow fishing net to the stick, unroll the net, and tie the other end to another stick. Sandro was incredibly deft rolling out the net and steering the boat with a paddle at the same time, which made it clear that he'd done this many times before. We hadn't known that fishing was part of the trip, so we had a lot of fun watching the entire process.

Shortly after that we arrived at our home base for the following two nights, the Guardiania Añangu inside Yasuni. We set up our tents in the special area they had for visitors -- a raised cement platform with a roof. The guardiania had water running constantly at high volume from a tap that was never shut off, probably because it was connected to a nearby spring. They also had a generator that provided electricity at night.

With only an hour of daylight left, we traveled by boat to a nearby mirador, a 42-meters high observation tower ascending above the treetops to afford a panoramic view. We climbed the tall spiral staircase and felt the tower wobbling in the wind, despite the steel cables that anchored it to the ground. The view from the top was breathtaking: a 360-degree ocean of green extending to the horizon in all directions. I've never felt as small as when I was up there looking at that vast, untamed landscape.

Green-feathered parrots were flying among the trees below us, abundant and unselfconsciously loud, as common here as crows or pigeons in a European city. We watched the sunset from the mirador, and then boated back to our tents. Using a portable gas stove, Sandro cooked a delicious dinner of lentils, rice, and fried chicken. We had two helpings each and settled in for the night.

Day 6: The Jungle

By the time the sun rose, we were already on the boat.

We went to check our fishnet, and found nine fish of five different species.

Sandro then took us to a clay lick -- a place where parrots go to eat clay, which helps their digestion. A dozen parrots were swarming around a particular spot on the shore, gurgling and chirping with enthusiasm.

After that we ate breakfast, laid the fishnet again, and went to another clay lick where we hoped to see macaws. This lick was deeper inside the forest, and there was a wooden rotunda with about a dozen other birdwatchers sitting and waiting. The lick was noisy with lots of small green parakeets. They looked like fairies with their bluish, fast-moving wings.

A scarlet macaw hung around the trees above the clay lick, hesitating to descend. When it finally did, all the parakeets fled the scene. The macaw was majestic, much larger than the parakeets, with bright red and yellow and blue feathers. Around this time, a vicious wind started blowing, and pretty soon we found ourselves under torrential rain. The weather had been great until then, and we were not expecting such a sudden change, so we had left our rain gear back at the tents. We waited for about half an hour, but the rain had more patience than us. So we ran back to the boat, getting completely drenched, but miraculously staying warm (this being Ecuador).

When the rain was over, we went to check our fishnet once more, and this time Raunak and I helped Sandro untangle the fish that got caught. I have to admit that when I held the fish in my hand, and felt it wriggle and squirm violently, my first instinct was to loosen my grip. But I did not, and I did not become vegetarian either.

That evening we went to the observation tower again, and saw a pack of howler monkeys, as well as many species of birds. At dusk, Sandro took us for a night walk in the jungle. We saw more monkeys while the sky was still lit, and then it got dark and scary and awesome. Our flashlights carved cones of light into the thick darkness, which was alive with cicadas and other noises that were harder to identify. On trunks of trees we saw spiders as big as my hand.

With his eagle's eyes, Raunak also spotted a stick insect:

For dinner that night, Sandro fried the fish we had caught earlier. They were much tastier than supermarket-bought fish, but they also had a lot of small bones to reckon with. As we ate, lots of insects swarmed to the bare electric bulb above our heads, but there were surprisingly few mosquitoes. Maybe we went there at a lucky time of year, but we barely got bitten throughout our trip.

Day 7: Level Tena

That night, a frog had decided to lodge itself on one of the roof beams of the structure we were sleeping under. All night we heard it talking to its relatives in the forest, the roof acting as a perfect amplifier.

In the morning we went for a final 3.5-hour hike through the jungle. Sandro spotted a tree with hollow stems in which lemon ants dwelled. He proceeded to open up the stems and eat the ants, which Raunak tried, but I didn't dare. We saw more monkeys, spiders, and even a forest turtle.

At the end of the trail we reached a watering hole for wild pigs. There were at least fifty of them, maybe even a hundred. We got to observe them in silence for a minute or two, and then they smelled us and bolted, with the sound of hundreds of hooves hitting the ground at the same time.

On the way back, we noticed paw prints of tapirs and jaguars on the trail. We saw more large butterflies, but most of them were too quick to catch on camera.

After that we spent 3+ hours on the long boat ride back to Coca. We said goodbye to Sandro, and decided to drive to Puyo during the night, so that we'd have less driving to do the following day. It was a good thing that Raunak was at the wheel, because the game of Ecuadorian driving had stepped up a few levels in difficulty. There was Level Speed Bumps, Level Bikes and Motorbikes without Lights, Level Taxi Coming in the Wrong Direction, and Level Dogs Sleeping in the Middle of the Road, all of which Raunak navigated expertly. But of course, when you pass a level, you get a harder level, and so we got Level Tena. This was a pretty big town that was completely missing from Google Maps, and where we got lost for about 20 minutes before we found the highway again. When we reached Puyo, we crashed at the first hotel that we saw.

Day 8: Llamas and Fog

After a long-awaited hot shower at the hotel, we drove to the city of Baños nearby. Baños was very touristy, with lots of private tour companies and little information about public trails. We got breakfast and decided to go to the Volcán Tungurahua directly south of the city. We drove up a dirt road until our car couldn't handle the slope, and from there we went on foot. The farmland around us looked strange; every greenhouse and every cow was tilted to accommodate the steep gradient of the terrain. We hiked a few more kilometers, the visibility getting worse and worse, until we found ourselves inside of a cloud.

With no hope of seeing anything more than 10 meters away, we turned back. We were eager to get to the coast, so we continued driving west. On Google Maps we saw a big blob of green called Reserva de Producción Faunística Chimborazo, so we decided to take the road through that. All went well on the highway, until we got into a maze of smaller and smaller roads. We persisted for about half an hour, because the Google directions promised to take us to a straight road very soon. But when we got there, we found that it was not a road at all, but a railroad. So much for trusting Google Maps in Ecuador! We went back to the highway, took a detour, and finally found the road through the reserve.

The terrain was getting increasingly gray and barren, like something out of Mordor. I checked my GPS and realized that we were above 4000 meters altitude. Raunak said that we should be seeing llamas, and sure enough, pretty soon we saw packs of them.

I got out of the car and walked closer to one, to take a picture. It was very cold, and even that short walk left me breathless because of the elevation. We reached 4400 meters before we started descending -- the highest on this trip, and also the highest in my life so far.

Coming down from the Andes we passed through another cloud, with fog so dense that it made driving a surreal experience. In the picture below there is a car ahead of us -- can you spot it?

Night fell fast, and pretty soon we had some serious level-12 driving conditions. In addition to the darkness, the rain, and the fog, there had been landslides that ate away at the edges of the road. We even saw a big excavator parked in the middle of our lane, with no lights or reflectors to alert us to its presence.

We stopped in Babahoyo and got dinner for $2.50. I replaced Raunak at the wheel, and we continued driving for a bit. I was fine with driving a manual as long as the car was moving, but if I stopped, I inevitably stalled the engine. This was especially entertaining when I stalled in the middle of the highway, trying to make a left turn... After going through a few small towns without hotels, we found one in Pedro Carbo, and stopped there for the night. In the past few hours, we had come down from an elevation of 4400 meters to about 14 meters.

Day 9: Playa de los Frailes

In the morning we continued driving towards Puerto Lopez. There was a promising road going through the Parque Nacional Machalilla, but we found that the pavement only went as far as the village of Julcuy. A friendly villager warned us that we couldn't pass through the park, because the rio was alto from the recent rains. Of course, we had to try anyway. We had two warning signs, in the form of big puddles of water on the road, where Raunak told me to go fast and get enough momentum to avoid getting stuck in the mud. Then we got to the rio proper: there was no bridge, and the water running across the road seemed at least chest high. There was no way our little car could make it through that, so we turned back.

On our way back we hit the boss level of the driving game. To avoid one of the puddles we had plunged through earlier, we steered off the main dirt road and into an area that looked drier. It only looked that way though, because our car quickly got stuck in mud. We got out and tried to push it from behind, but all that did was to sink our feet and our tires deeper into the mud. We picked some rocks and put them under the wheels, hoping to get more friction, but all we got was the smell of burnt tire. For about half an hour we were stuck in place, revving the engine like crazy, with streams of mud spurting from the rear wheels and falling all over the place.

We finally figured out that we could inch forward by turning the wheel all the way left and then all the way right, and revving the engine. When we got out, our car had chunks of mud clinging to every surface. Raunak decided that his shoes were beyond salvaging, so he just left them there. Mine were so filthy that I put them in a plastic bag and kept them in the trunk. We spent the rest of the day in flip-flips.

Cured of our desire to take dirt roads through national parks, we reached Puerto Lopez by highway. We ate lunch (at US prices), watched the auto rickshaws / mototaxis that were so popular here, and booked a tour of Isla de la Plata for the following day. Raunak bought new shoes. And then we drove to the nearby Playa de los Frailes.

For me, this was one of the highlights of our trip. I normally can't handle beaches in the daytime, because I get sunburnt in about five seconds. But on this day, there were thick clouds shielding us from the sun. The water was warm, almost as warm as the air. The beach was beautiful and nearly deserted. No seashells to cut your feet on, no seaweed or jellyfish to wade through, and no commercial development, since it was part of the Machalilla national park. The waves were pretty high, and we played the game of breaking them before they broke us. The hour and a half we spent in the water is inscribed in my memory as a time of undiluted joy, the kind I used to feel when I was a kid.

After that we headed to the village of Agua Blanca, hoping for another way to enter the Machalilla national park. The village had an observation tower with glorious views of greenery all around. But when we tried to take the trails into the park, the same rio thwarted us again, the amount of water making passage impossible. We ended up camping in the village, under a special wooden structure they had for visitors.

Day 10: Isla de la Plata

Agua Blanca was quiet... except for the roosters crowing, dogs barking, pigs grunting, mules neighing, goats bleating, people chatting, and motorcycle engines revving all night. In the morning we wanted to go to the Los Frailes beach again, but the entrance road was closed. So we went to Puerto Lopez, had piña coladas for breakfast, and waited for our boat to Isla de la Plata.

We ended up getting separated for the tour. After 90 monotonous minutes on the boat, we reached the island. I was surprised at how dry it was, not at all like the lush mainland. My group went on a frustratingly slow hike, with a guide who stopped too often to tell us about the folk-medicinal properties of every plant there. (I understood just enough of his Spanish to know what was going on, but not enough to ask any questions.) The island had plenty of interesting birds: elegantly shod blue-footed boobies and obscenely tumescent magnificent frigatebirds.

After the hike we went snorkeling. The water was clear and warm, and there were colorful fish swarming underneath. When I dove fast enough I could almost grab one by the tail. I kept getting stung by tiny jellyfish that I couldn't see. The bites weren't too painful, until one of them stung me on the lip.

When we got back to shore, it was time to say goodbye to the coast and head back to Quito for our flights the following day. We stopped at a hotel in Portoviejo for the night. The guy at the front desk said we could park in front of the building, but when he actually saw our car, he did a double take. And then he offered to have someone wash it for us for $1. By the time that was done, there were four little mountains of mud underneath the fenders.

Day 11: Don't go to Quito

With less than 12 hours until our flights and more than 500 kilometers left to go, we gunned it straight towards Quito. We were worried about potential last-minute delays, especially when we hit a stretch of highway that was under construction and full of potholes. But even with Level Potholes, Level Mist, and Level Cocoa Beans (drying on the road because it was the only available flat surface), we made it to the capital in plenty of time.

We stopped at the Metro del Sur Park just south of Quito, but there was little to see there in the chilly drizzle. So we decided to go to the Old Town and check out some of the architecture. After so many days spent in wide open nature, the city felt cramped and claustrophobic, like a shirt that's too tight and won't let you stretch. Some of the narrow cobblestone streets were so steep that we thought our car would start rolling backwards at any minute.

After walking around the Old Town for a bit, we headed towards the airport. Traffic was miserable by this time, and the exhaust from all the idling cars was suffocating. (It made me appreciate the strict air quality standards in California.) Quito's roads are a merciless maze of roundabouts and underpasses, and even with the help of Google Maps, it took all of our attention to stay on course.

After eleven days and 2335 km (1,451 miles) traveled, we couldn't quite believe that our trip was over. From the endless greenery of the rainforest, to the thin air of the highlands, to the beautiful beaches on the coast, Ecuador has made a place for itself in my heart. I hope to be back someday. As for Raunak, he's already signed up for Spanish classes.

Julia Profiling Cheat Sheet

2015-05-30T00:00:00-07:00

I've been using Julia for all my research code in the last few months, and I love it. It has many of the same features that make Python a pleasure to use: concise code, high-level abstractions, a powerful REPL, and a notebook interface. On top of that, Julia has some features that I missed in Python: a powerful type system, and the ability to run at C-like speeds if you write your code carefully.

Good profiling tools are essential for optimizing code, and I wrote in the past about profiling in Python. In this post I will summarize Julia's profiling tools, and a few tricks and gotchas that I learned along the way. I assume familiarity with the sections on profiling and performance tips in Julia's manual.

Cheat sheet: Timing macros

@time prints timing and allocation info, and returns the expression's value:

julia> @time "the answer is $(2*3*7)"
elapsed time: 1.0246e-5 seconds (480 bytes allocated)
"the answer is 42"

@elapsed returns the seconds elapsed, and discards the expression's value:

julia> @elapsed "the answer is $(2*3*7)"
1.0404e-5

@allocated returns the bytes allocated, and discards the expression's value:

julia> @allocated "the answer is $(2*3*7)"
400

@timed returns a tuple of (expr_val, sec_elapsed, bytes_alloc, sec_in_gc). This macro is undocumented, so it might disappear in future versions of Julia.

julia> @timed "the answer is $(2*3*7)"
("the answer is 42",2.6062e-5,480,0.0)

Cheat sheet: Profiling

For long runs, reduce the profiler's sampling frequency to ensure that we have enough space to store all the backtraces:

Profile.init(delay=0.01)

Run the code once to force compilation, then clear any previous profiling results, and run the code again with profiling:

code_to_profile()
Profile.clear()
@profile code_to_profile()

Visualize results:

using ProfileView
ProfileView.view()

Cheat sheet: Tracking heap allocations

Start Julia with --track-allocation=user to track mallocs in your code, or --track-allocation=all to track mallocs in your code as well as Julia's libraries.

Run the code once to force compilation, clear previous info about memory allocations, then run the code again:

code_to_profile()
clear_malloc_data()
code_to_profile()

Then quit Julia and inspect the *.mem files to see which lines cause heap allocations. (The *.mem files will not be written out until you quit Julia.)

Trick: Saving profile results to disk

Sometimes you want to store profiling results for later comparison. Other times matplotlib interacts poorly with ProfileView, and you are forced to visualize profiling results from another instance of Julia.

To save profiling results, we grab them from the Profile module and dump them to a file:

r = Profile.retrieve();
f = open("profile.bin", "w")
serialize(f, r)
close(f)

Or as a one-liner using a do-block:

open("profile.bin", "w") do f serialize(f, Profile.retrieve()) end

To load profiling results and visualize them:

using ProfileView
f = open("profile.bin")
r = deserialize(f);
ProfileView.view(r[1], lidict=r[2])

Or as a one-liner:

r = deserialize(open("profile.bin")); ProfileView.view(r[1], lidict=r[2]);

Gotcha: The JIT compiler causes memory allocations

Julia code is just-in-time compiled, so the first call to a function does a lot more work than subsequent calls. You probably don't want to measure the time and memory used by the compiler, so run your code once to force compilation, and then run it again to profile it. Example:

julia> function foo() sum(ones(5)) end
foo (generic function with 1 method)

julia> @time foo()
elapsed time: 0.004021931 seconds (23432 bytes allocated)
5.0

julia> @time foo()
elapsed time: 5.696e-6 seconds (192 bytes allocated)
5.0

For the same reason, it is best to avoid include() calls in the code that you are profiling. Instead, include everything you need beforehand.

Gotcha: The REPL causes memory allocations

Code written in the REPL is evaluated in the global scope, and we already know that globals can cause slowness. Additionally, code executed in the REPL causes memory allocations in the REPL itself, which you probably don't want to count. Example:

julia> function foo() 1 + 2 + 3 end
foo (generic function with 1 method)

julia> foo(); @time foo()
elapsed time: 4.217e-6 seconds (80 bytes allocated)
6

julia> function benchmark() @time foo() end
benchmark (generic function with 1 method)

julia> benchmark()
elapsed time: 2.97e-7 seconds (0 bytes allocated)
6

Gotcha: Anonymous functions are not properly optimized

I ran into this issue when writing FactCheck tests. The standard syntax uses do-blocks, which create anonymous functions. Unfortunately, anonymous functions are not optimized as well as regular functions in current versions of Julia, so they may run more slowly and allocate more memory. Example test file example.jl:

using FactCheck

# Define an immutable type, which Julia allocates on the stack.
immutable Point
    x::Float64
    y::Float64
end

function Point(arr::Array{Float64,1})
    @assert length(arr) == 2
    Point(arr[1], arr[2])
end

# Ensure that the Point() constructor is compiled.
let
    Point([3.0, 4.0])
    println("compiled")
end

facts("test with anon func -- will fail") do
    arr = [3.0, 4.0]
    bytes = @allocated Point(arr)
    @fact bytes => 0
end

function test_func()
    arr = [3.0, 4.0]
    bytes = @allocated Point(arr)
    @fact bytes => 0
end
facts(test_func, "test with regular func -- will pass")

Now from the REPL:

julia> include("example.jl");
compiled
test with anon func -- will fail
  Failure   :: (line:-1) :: got 24
    bytes => 0
Out of 1 total fact:
  Failed:   1
test with regular func -- will pass
1 fact verified.

Bonus: My all-in-one recipe for profiling

Putting it all together, my profiling code follows the basic recipe below. It prints timing and allocation info to stdout, dumps the profiling data to profile.bin, and also produces *.mem files if Julia was started with --track-allocation=....

function benchmark()
    # Any setup code goes here.

    # Run once, to force compilation.
    println("======================= First run:")
    srand(666)
    @time code_to_profile()

    # Run a second time, with profiling.
    println("\n\n======================= Second run:")
    srand(666)
    Profile.init(delay=0.01)
    Profile.clear()
    clear_malloc_data()
    @profile @time code_to_profile()

    # Write profile results to profile.bin.
    r = Profile.retrieve()
    f = open("profile.bin", "w")
    serialize(f, r)
    close(f)
end

Trip to Bishop, CA

2015-05-30T00:00:00-07:00

Bouldering + hot springs + mountain biking = best weekend ever.

(more pictures)

Seeing Nightwish Live

2015-04-29T00:00:00-07:00

Nightwish has been the soundtrack of my teenage years. I have since drifted away from that kind of music, because I think it reinforces a certain sadness. But when I heard that they were coming to San Francisco, I decided I had to see them live at least once. I'd never been to a concert like this before, partly because I'm cheap, partly because crowd behavior freaks me out, and partly because live songs never attain the perfection of their album versions. But I'm so glad I went! The climax of the evening came for me when they played Sleeping Sun -- an old-time favorite that I hadn't heard in years. I had a Ratatouille moment that brought me back to a particular time in Moldova, when I got up early to run around the lake and watch the sun rise, and I had listened to this song. I discovered that I still knew all the lyrics, and apparently, so did the people around me. Everyone sang along, and the entire room shivered with memories.

Becoming Solid

2015-04-11T00:00:00-07:00

"I'm just a collection of mirrors, reflecting what everyone else expects of me." Rollo May begins his book, Man's Search for Himself, with the above quote from one of his patients. His writing style is a bit circuitous, and trying to summarize his ideas is like trying to find sharp contours in a blurry photograph, but I'm going to try anyway. The problem he is addressing has the following manifestations:

Feeling like your life is empty, futile, and meaningless.
Feeling like you are a pawn in the hands of external forces, and have no power to direct your own life.
Thinking that the world is going to hell and you are too small and insignificant to influence anything.
Feeling lonely, afraid of being alone, and terrified of being a social failure. Feeling a compulsive need to be liked and approved of.
Feeling a vague but constant anxiety, and being unable to pinpoint its source.
Thinking that your goals and circumstances could change at any moment, and that there is too much uncertainty about the future, so there is no point starting any long-term projects.

May's proposed solution is to develop a stronger sense of self, and to dare to stand on your own two feet. Having a strong sense of self means being aware of what you feel and what you want. The opposite of this is pretending that you never feel envy or anger, and only wanting things that you're willing to admit in polite company. Standing on your own two feet means making choices that you believe are right and worthwhile, and taking full responsibility for those choices, even when they might annoy certain people. The opposite of this is picking an audience (parents, teachers, friends, or the anonymous crowd), and making choices based on how that audience would react.

Both prongs of this solution require a fair bit of courage: the courage to look inside yourself and risk seeing things you'd rather not see, and then the courage to go against what is expected of you and towards what you really want, risking embarrassment and disapproval. It is much easier to abdicate responsibility, to live your life in perpetual reaction to external demands, to keep yourself constantly busy and avoid self-awareness... To dream about a better tomorrow, forgetting that life is just today over and over and over again, and that if you can't be happy with today, then tomorrow won't be any better.

When thinking about a person with a weak sense of self, the image I have in my head is that of a baby fish. She is taken wherever the currents take her, and has no will of her own. She has cartilage for bones. She's even slightly transparent, trying so hard to blend in that she takes on the hues of whatever is around her. As she gets older, the fish becomes more solid. Her bones establish a clear boundary between her self and what's outside. She is now strong enough to swim against the current. And she is no longer swayed by the colors of those around her. This is what it means to be a confident adult, to have a will of your own, to stand on your own two feet and be the master of your fate.

After outlining the problem and sketching the solution, May stops short of giving practical advice on becoming solid. I would have liked to see examples of people developing their self-awareness, courage, and integrity, and growing up from powerless to self-assured. Heck, even figuring out what you want is not an easy question, when you've spent all your life doing what other people expected of you. That question requires more thinking, and could be the subject of another post.

Jigsaw Puzzle Time-lapse

2015-01-04T00:00:00-08:00

This is what I did over winter break:

Details below, for the curious.

I used a Raspberry Pi with a camera module, stuck to the ceiling with double-sided tape, and about six different cables and extension cords, supported by a shoddy frame made of PVC pipes. I connected the Pi to my laptop via Ethernet, and mounted my laptop's hard drive on the Pi using sshfs. I recorded the video on the Pi with

raspivid -v -fps 10 -t 0 -o vid-part1.h264

and immediately played the growing file on my laptop:

tail -c +0 -f vid-part1.h264 |mplayer -cache 8192 -

This allowed me to see the video as it was being recorded, with only about a 0.5-second delay.

Assembling the jigsaw took 10.5 hours over three days. So the video above is playing about 140x faster than real time.

Then came the annoying part: Processing 54 GB of video. I know approximately nothing about video processing, so I don't recommend that anyone do this at home. I'm just recording my steps so that I remember how to do it next time.

First I dropped 9 in every 10 frames:

avconv -i vid-part1.h264 -filter:v "setpts=0.1*PTS" -vcodec libx264 out1.mkv

This took a few hours, and I was left with a number of short clips. I concatenated them and cut some dead time using openshot, then exported the result. Then I dropped some more frames to make the video the length I wanted:

avconv -i tmp.mkv -filter:v "setpts=0.5414*PTS" -vcodec libx264 out-video-only.mkv

Finally I loaded the soundtrack in audacity, adjusted the tempo a tiny bit so that the music and the video were the same length, and merged the two:

avconv -i out-video-only.mkv -i firestarter-stretched.mp3 -vcodec copy -acodec copy out-final.mkv

That was fun :)

How to Combine a Bunch of Images into a PDF File

2014-11-30T00:00:00-08:00

I will describe two methods: one using convert and one using pdfjoin.

Method 1 is to use the convert tool from ImageMagick. Advantages: This is simple, and you probably already have ImageMagick installed. Disadvantages: This will fail when you have a lot of large images, because it will try to load them all in memory at the same time.

Basic usage:

convert *.png output.pdf

If that uses all your machine's memory, you can force it to swap to disk when it exceeds a given amount of memory:

convert -limit memory 2GiB -limit map 4GiB *.png output.pdf

If that fills up your /tmp, you can tell it to swap to some other location:

MAGICK_TEMPORARY_PATH="/media/bigdisk" convert -limit memory 2GiB -limit map 4GiB *.png output.pdf

Unfortunately, if you have a lot of large images, swapping to disk is slow, and the command above gave me occasional segmentation faults. But if you only have a few images, convert is the easiest method.

Method 2 is to use the pdfjoin tool from the pdfjam package. Advantages: This works fine in cases where convert runs out of memory. Disadvantages: pdfjam uses LaTeX, and installing it will pull in about 500MB of dependencies.

First you have to convert your images into one-page PDFs:

for i in *.png; do convert $i $(basename $i png)pdf; done

Then you can join those PDFs together into one file:

pdfjoin --fitpaper 'false' --rotateoversize 'false' *.pdf -o output.pdf

This works well, without running out of memory, even if you have hundreds of high-resolution images.

Visualizing the Results of Profiling Python Code

2014-11-30T00:00:00-08:00

If you've ever profiled a Python program, you probably know about the cProfile module in the standard library. Examining the output of cProfile is not easy, as it requires you to remember incantations like p.sort_stats('time').print_stats(10). And if you see that a lot of time is spent in some function deep inside numpy, you'll have a hard time figuring out who exactly is calling that function. Fortunately, there are some good visualization tools out there. I will describe two of them: pyprof2calltree and pyinstrument.

pyprof2calltree allows you to visualize cProfile results using KCacheGrind, which might be familiar from your C days. Install it with:

pip install pyprof2calltree

Profile your program as usual:

python -m cProfile -o prof.out mycode.py

Then visualize the results in KCacheGrind:

pyprof2calltree -i prof.out -k

This gives you a visualization of the call graph, and how much time is spent in each function:

The other tool, pyinstrument, works entirely on the command line. Install it with:

pip install pyinstrument

Run your program like this:

python -m pyinstrument mycode.py

And you get a nice color-coded call tree that tells you where the time is spent:

I really like the simplicity of pyinstrument, and I think it will become my go-to Python profiler from now on. The only problem I noticed was that pyinstrument seems unable to track time spent in C-extension calls, so if you have any Cython code, you will not see how much time is spent there. (cProfile + pyprof2calltree don't seem to have this limitation.)

Viren: Bulk-Rename Files Using Your Editor

2014-10-09T00:00:00-07:00

Often I want to rename multiple files in a similar way. There are just enough files that the repetition is annoying, but not enough to bother starting a GUI bulk-renamer like krename and configuring it to do what I want. I know how to avoid repetitive work in my editor, so it would be nice if I could just rename the files from there.

I wrote a little tool called viren that enables this. When launched, viren opens an editor window with the list of files in the current directory. You can use your editor's features to change the file names. (But don't add, remove, or reorder the lines.) When you save and quit the editor, viren will rename the files as you requested.

Here is a quick demo. I have a project called crop_many. I want to rename it to RotateMany, and also change the capitalization from snake_case to CamelCase. I can do this quickly with viren:

To install viren, do pip install viren.

Note that viren calls editor internally, so it works with any editor, not just vim. On Debian / Ubuntu, you can choose which editor to use with update-alternatives --config editor.

Shell Shortcuts for Recently Downloaded Files

2014-09-17T00:00:00-07:00

I often find myself downloading a file in the browser, then switching to the terminal and moving the file into my current working directory, where I do something with it. Often I forget what the file was called, so I have to switch to my browser one more time just to check the filename. This got annoying enough that I wrote some bash shortcuts to help me.

mvdl will move the most recently downloaded file into the current directory. To get the second most recently downloaded file, use mvdl 2. To specify a destination other than the current directory, use mvdl 2 dest.

cpdl is just like mvdl except that it copies the file instead of moving it.

lsdl displays the 10 most recently downloaded files, so that you know what number to supply to mvdl or cpdl.

Here's the full bash snippet to make this work. Let me know if you have suggestions for improvement.

There Might Not Be An Elegant Solution

2014-09-02T00:00:00-07:00

On a recent trip to London I visited the National Maritime Museum, which had an excellent exhibit on determining longitude at sea. I didn't know this, but finding a ship's longitude was one of the biggest scientific puzzles in history. The British Parliament in 1714 instituted a prize for solving this problem, although people had been thinking about it long before then. Two practical solutions emerged eventually: the lunar distance method and marine chronometers. Both solutions kept track of time at a fixed point -- Greenwich. Navigators then determined longitude by the difference between the ship's local time and Greenwich time.

The lunar distance method measured the passage of time using the position of the moon with respect to other celestial bodies. There was no simple formula describing the movement of the moon with sufficient accuracy, so navigators relied on nautical almanacs, which were basically lookup tables precomputed by the government. Such almanacs were published starting from 1767, and required teams of human computers to carry out the calculations every year.

Marine chronometers were the first clocks accurate enough to keep time on a moving ship. The first such clock was Harrison's H4 in 1761. It had an incredibly complicated mechanism, including a 5 Hz mechanical oscillator, a bimetal strip to correct for temperature variation, and clever machinery that allowed the clock to keep running while it was being rewound. H4 took six years to build, if you discount the two decades Harrison spent on its predecessors. Marine chronometers took until ~1825 to become simple enough to be manufactured at scale.

Solving the problem of longitude took decades of focused work, and the dedicated careers of multiple scientists and clockmakers. Though the problem is simple to state, its first practical solutions required vast amounts of painstaking calculations, tinkering, and engineering. I often look at a problem and think that it ought to have a clean, elegant solution, and that anything less parsimonious than that is not worth spending time on. I hear the same bias from other people, when they speculate that progress in neural nets / solar cells / insert-favorite-subfield-here will solve AI / green energy / insert-favorite-problem-here. There might not be a shortcut. There might not be an elegant solution. Solving the problem might just require a ton of detailed effort and engineering. And the solution might take longer than the time scale of a single person's career.

Sauvie Island: A Tale of Stupidity and Kindness

2014-08-10T00:00:00-07:00

On a recent trip to Portland, Oregon, I rented a bike and spent the weekend exploring. Here is the true story of what happened on Sunday.

This is my first time on a proper road bike, and everywhere I go feels like downhill. I got a patch kit the other day, just in case. But I didn't get a pump, because I thought there would be enough bikers around me to lend me one if I needed it. You already know where this is going, right?

Today's destination: Sauvie Island. The weather is warm but not sunny; pretty much perfect by my definition. My first stop is a pick-your-own-berries farm. You just pick whatever you want, and then pay for it by weight. I take a box and walk into the well-tended rows of blue/black/raspberries. For every berry I place in the box, I put three in my mouth. This reminds me of rummaging through my grandma's raspberry bushes when I was a kid, and I am happy. When my stomach says it's had enough, I placate it with water, and go pay for my box. It's only a couple of dollars, and I think the entertainment itself was worth more than that. I get back to my bike, and decide to do a 12-mile loop around the island before heading back to my hotel.

Midway through the loop, I get a flat tire. At this point, the uselessness of my patch kit becomes apparent. Without a pump I can neither find the hole, nor inflate the tube after I patch it. And of course, I'm in the middle of a sparsely populated island, and I haven't seen any bikers in a while. Now: when I'm in perfectly safe situations, I find 1,001 things to worry about. But when I'm stranded in the middle of nowhere with a flat tire, I just laugh at the craziness of it all. And I keep waiting to see what happens next, as if this were not my life but some story I was reading. Maybe it's because I so rarely find myself in genuinely threatening situations, that my mind only gets so far as "Is this really happening to me?"

I start walking along the road, thinking that I'll see a farmhouse eventually, and maybe they'll give me a ride. After a few minutes, I see a pickup truck coming along the road behind me. I raise my thumb and it stops. Three guys inside. I explain my predicament, put on the most sheepish smile I can muster, and ask if they'll give me a ride to the bridge off the island. "Sure, hop on." And so I get to see the rest of Sauvie Island riding in the back of a pickup truck, with my sunglasses on and the wind sweeping past. Which is so much fun, I can barely wipe the grin off my face.

The full saltiness of my pickle doesn't become apparent until the guys drop me off and I cross the bridge to NW St Helens Road, along which I came. A few hours ago there were lots of cyclists on this highway, doing the Seattle-to-Portland bike ride, but now I can't see a single one. I raise my thumb again, but of course nobody takes hitchhikers on the highway. Bus 16, the only bus along this road, doesn't run on Sundays. I could call a cab, but I doubt they'll take me and my bike. My other option is to walk 4 miles south to St John's bridge, cross into North Portland, and catch a bus from there. I do a quick calculation of the remaining daylight, and decide that I can make it to the bridge before dark, if I hurry.

Jogging along the highway and rolling my bike along, with 60MPH traffic by my side, it finally occurs to me that this might not end well. I have a vision of being stuck on this road, in the dark, exhausted and cold, with no water left and a dead battery on my phone. Still, there's only one thing to do. So I keep on jogging. After about 1.5 miles, I see a pair of cyclists going the other way, and I shout to get their attention. (It's amazing how shyness evaporates when I'm in real need.) They stop. I wait for a lull in the traffic, and cross. Yes, they will lend me their pump and wait while I patch my tube.

Except I can't find the fucking hole. I remove the tube and pump some air into it, but the highway is too loud to hear where the air escapes from. And I can't feel it on my skin, either. I'm also in a hurry because these guys are waiting for me. If only I had a bowl of water to submerge the tube in, then finding the hole would be easy. At this point, a big dude on a motorcycle stops by, and asks us what sort of trouble we're in. I explain the situation, he takes out his knife, and proceeds to calmly cut his plastic water bottle in two. With that, I quickly find the hole, patch the tube, and put everything back together. I proffer abundant thanks to all of them (and also to whichever genius came up with instant vulcanization) and I'm on my way. I get to my hotel without further incidents.

The lesson, kids, is that you should always have a patch kit AND a pump, whenever you're biking out of the city. But there was another lesson in this for me. I've spent most of my adult life aiming to be completely self-sufficient, perhaps because trusting people isn't exactly my strong suit. And on this day I got myself in such a mess that I had to rely completely on the kindness of strangers (six of them, no less) to get me out. Maybe there is hope in the world, after all.

(image by delovelyArts on etsy)

The Black Swan

2014-06-01T00:00:00-07:00

Taleb divides the world into three kinds of events:

White swans. This is "mild" randomness, where one new measurement cannot affect your sample average too much. This applies to quantities like a person's height or weight, where large deviations are physically impossible. The Gaussian distribution works well for white swans, "taming" their randomness and yielding predictions that we can be confident about.
Gray swans. This is "wild" randomness of the tractable kind. One new measurement can dwarf all previous ones, thus moving the average quite far. This applies to socially constructed quantities such as book sales for a sample of authors, or net wealth for a sample of people. We can model such distributions using power laws, but we should be wary that the exponent of a power law is more difficult to estimate from data than the parameters of a Gaussian. Taleb argues that we should use power-law models only to describe phenomena, not to make concrete predictions.
Black swans. This is "wild" randomness of the intractable kind. Events in this category represent the unknown unknown, e.g. a new technology that changes the world, a sudden war after a long period of peace, and so on. Anything that is unexpected but has extremely high impact is a black swan.

In Taleb's colorful terminology, white swans live in Mediocristan, while gray and black swans live in Extremistan. His central thesis is that we almost always assume a variable is in Mediocristan, when in fact it's in Extremistan. Taleb is an options trader who made a lot of money in several stock-market crashes by betting on extremely unlikely events. As such, he has a lot of distaste for economists and financial forecasters, whose prediction track record is extremely poor, though no one notices this (more about this in a minute). Taleb accuses these forecasters of "Locke's madness": reasoning correctly from erroneous premises. He claims that their fancy models rest on white-swan, Gaussian variables, and are thus completely useless in a world dominated by gray- and black-swan events.

Why do gray and black swans rule the world? This has to do with the idea of scalability: If a book sells a thousand copies, then selling the one-thousand-first copy requires no extra effort from the writer. But if a baker sells one thousand cookies, the one-thousand-first cookie still requires work. So the writer's profession is scalable, whereas the baker's is not. Scalable phenomena exhibit winner-takes-all effects, thus giving rise to a power-law distribution. (There is one J. K. Rowling and she sells her books everywhere.) Non-scalable phenomena are necessarily local. (Every city has its own bakery.) Technologies such as the alphabet, the printing press, and sound recording enable scalability, and so they make our world less like Mediocristan and more like Extremistan. But our brains evolved in Mediocristan, and are ill-equipped to deal with scalable phenomena.

In what ways do we pretend that black swans don't exist? Here Taleb describes a constellation of fallacies and heuristics that make us less rational than we'd like to believe we are:

Confirmation bias. We tend to overvalue confirming evidence, failing to realize that no amount of it can prove a hypothesis. We tend to ignore or disqualify falsifying evidence, even though a single piece of it is enough to prove our hypothesis false.
The narrative fallacy. Our memories require dimensionality reduction, so we weave events into stories, making them easier to remember. In the process, we keep the details that match the story, and forget everything else. We invent causes for events, when the real causal graph might be more complicated, or even unidentifiable. (My cognitive psychology professor liked to say that we humans are obsessive pattern seekers, happy to see patterns even where randomness is at work.)
Hindsight bias. In retrospect, it seems inevitable that WW2 would have taken place. We can find lots of confirming evidence, and weave a convincing story. But if you look at the writings from that time, no one knew such a big war was coming.
The various biases and heuristics that we use in day-to-day decision making. For example, we give more weight to anecdotal evidence, especially if it has a vivid story (a plane crashing into a skyscraper) than we give to statistical evidence (more deaths per mile traveled in a car than in a plane).
Silent evidence. We only hear those who lived to tell the story. For example, "beginner's luck" in gambling is explained like this: Those who had bad luck at the beginning did not take up gambling; therefore a disproportionate fraction of gamblers did, in fact, have luck at the start. Silent evidence gives us an illusion of stability and safety: We underestimate the risks we took in the past, because we survived them all. If we hadn't, we wouldn't be here to contemplate the question. So we compute odds from the point of view of the winning gambler, and not based on everyone who started in our cohort.
The ludic fallacy. We assume that the world is like a game of chance, where the probabilities are known and well behaved. But the world if full of high-impact risks that we don't even model.
Epistemic arrogance. We overestimate what we know, and underestimate what we don't know. We fail to appreciate that the forward process (starting at A and observing B) is much simpler than the backward process (observing B and inferring that its cause was A). Historians think that they understand causes, but Taleb doesn't trust them.
Self-serving bias. When our prediction turns out to be right, we attribute our success to our skills and insight. When our prediction turns out to be wrong, we blame our failure on external circumstances, unforeseen events, and outliers. This bias allows us to be blind to how bad our prediction track record truly is.
Platonification. We cut the world up into crisp categories, and then assume that the world matches our model precisely. (If it's not in my model, it doesn't exist.)

So what are Taleb's take-home lessons? Don't trust the forecasters. Train yourself to take important decisions rationally, not intuitively. Focus on the consequences of an event (which are knowable), and not on the probability of the event (which is unknowable except in Mediocristan.) Maximize your exposure to positive black swans (go to parties, seize anything that looks like an opportunity). Remain paranoid about negative black swans.

This book was a lot of fun to read. Taleb combines his insights with his life story, and plenty of personal philosophizing. His tone is at times rambling and arrogant. Here is a surprisingly bitter quote from chapter 14:

Is the world unfair? I have spent my entire life studying randomness, practicing randomness, hating randomness. The more that time passes, the worse things seem to me, the more scared I get, the more disgusted I am with Mother Nature. The more I think about my subject, the more I see evidence that the world we have in our minds is different from the one playing outside. Every morning the world appears to me more random than it did before, and humans seem to be even more fooled by it than they were the previous day. It is becoming unbearable. I find writing these lines painful; I find the world revolting.

Nassim Nicholas Taleb in The Black Swan

If you could handle that, you can handle the whole book. Seriously, most of it is not nearly as bitter. Or you could grab his first book instead, which I haven't read. Some reviewers on Amazon think it's less rambling.

Visiting Vancouver

2014-04-15T00:00:00-07:00

The border guard towers over me, unsmiling, and asks me what my business is in Canada. He swipes a thick black line through my single-entry visa, but he lets me through. Apart from that, it doesn't feel like I left the US at all. American chain stores, American brands in the supermarket, signs in English only. Money in politics here, too, in the way real-estate developers control the city. According to my airbnb host, anyway.

I rent a bike and go to Stanley Park. It's cloudy and it drizzles, but that bothers me less than I expected. The tree trunks are dressed in moss, and everything is impossibly green. Swamp lanterns bloom on the edge of the path. After a year in droughty California, my eyes feast on this water-rich forest landscape.

One day is dedicated to suspension bridges. Capilano is impressive but crowded. The one at Lynn Canyon is breathtaking because it disappears into a wall of green (photo above). A waterfall is roaring underneath. I am amazed that I could get here with public transportation.

I take the ferry to Victoria, and it's the hugest ferry I've ever seen. Hundreds of cars, and even a few buses and semis roll into its belly. I go to the Butchart Gardens and reflect on the self-reinforcing nature of wealth. Then I go to Miniature World and watch in wonder as the tiny trains whiz by on their tiny tracks. On the way back, I read about the ghost towns of British Columbia. Another trip is in order for those.

Balance

2014-04-07T00:00:00-07:00

This is a good time to think about balance. A student in my department committed suicide two weeks ago, leaving behind a heartbreaking poem about hard work and isolation. (I did not know him.) The department's response was dismal: They're organizing a symposium to celebrate his life and "short but splendid research career". The silence and stigma associated with depression and suicide are not helping the problem -- they are making it more difficult for affected people to seek help. What we need is an earnest conversation about mental health, as advocated by organizations like Active Minds. We need to stop being ashamed to talk about it. We need to know what warning signs to look for in ourselves and in others. And most of all, we need to know how to seek balance in our own lives. But alas, work-life balance is often seen as an euphemism for slacking off, in an environment where we see colleagues routinely putting in 80-hour weeks. Grad school is not for the faint of heart.

Wasted Time

2014-04-02T00:00:00-07:00

At some point I lost the ability to be at rest. Dreading wasted time, I felt the impulse to fill every minute of my day with something productive. And when I took a break to watch a movie or to read for pleasure, I often felt guilty, knowing that I would reach the end of the day with my to-do list still unfinished. I thought that the more I achieved, the happier I would be, so I optimized my life for efficiency.

But efficiency meant that certain things fell by the wayside, "non-critical" activities like going to the gym or hanging out with people. After a few years, the effects of this policy became clear: I felt isolated and unable to connect with other people, and I started having repetitive-strain problems with my hands. The upside? I graduated top of my class, and I got into an excellent grad school. It was a Pyrrhic victory, in retrospect, given that my original fascination with computer science was quashed by the drudgery of never-ending problem sets.

I am still very much in search of balance. I recently picked up Carl Honoré's book In Praise of Slowness, hoping that it might provide some sort of guide. The book acknowledges our obsession with doing everything faster, and its corollaries: not eating well, not getting enough sleep, feeling burnt out at work and impatient at home. The book then enumerates various things that people and communities do to slow down, to maximize enjoyment rather than efficiency. These include farmer's markets, walkable neighborhoods, meditation, yoga, part-time work, knitting, gardening, and so on. The common theme here is a desire to unplug from the 24/7 machinery of globalized capitalism, and carve out some time for calm and relaxation.

But the book left me unsatisfied. It merely scratched the surface of the underlying questions: Why is unstructured time uncomfortable, like a vacuum to be filled? Why are we drawn to speed, efficiency, and workaholism? Why do we always feel like we're running out of time? I think the answer has to do with our tendency to measure self-worth in terms of achievement, and the habit of comparing ourselves to other people. David Burns has some interesting things to say about these in his book Feeling Good.

My own drive to work hard and be efficient came from a poorly articulated fear that if I slowed down, if I didn't work as hard as I could, then I would somehow fail, and be unable to live the kind of life that I wanted. Exposed to the light of reason, that belief is clearly irrational. I'm finally starting to learn how to take control of my life, and not be a slave to my to-do list anymore.

Happiness Is

2014-03-24T00:00:00-07:00

For three days the rain keeps hammering down with no sign of mercy. Birdsong and color are swallowed by the downpour, and the world becomes a Nick Cave song. Dead trees raise bare arms in supplication, but the sky is thick and callous like elephant skin, and the same color. My sneakers squeak against the slippery asphalt, leaving no trail, and following none either. It hasn't rained like this in a long time, and my mind wanders...

It wanders back to two winters ago. We're walking home from the T station. Our breath forms white clouds in the cold air. Your hand in my hand in my pocket, you say that they should make a glove for two, a glove so people could hold hands in November. Our shoulders bump as we try to fit under the tiny umbrella, which is mostly useless anyway, because the rain comes at us horizontally. Around us, neat Somerville houses flock together, duck their heads, and fluff up their feathers against the coming onslaught of winter. We climb a creaky staircase, and then we're finally inside. The tea smells good. My fingers tingle and come back to life. I reach for the Wired on your coffee table, but you lunge for it first, and then you tell me I'm only dating you for the magazines.

Your room is the lovely mess that it always is. A half dozen unfinished books glance up hopefully as we come in. A box of yarn winks from under the table. From the opposite corner, a papier-mâché lamp pours warm light over everything. The rain seems harmless and muted, banished outside your windows. Your bedspread is magenta, and your papasan is magenta, and your little-chair-that-is-a-box is magenta. And you're wearing that magenta long-sleeve that looks so pretty on you. And then we're under the blankets, and it's warm, and that long-sleeve comes off, and then my memory just kind of melts...

Happiness is: Waking up next to you.

The blue morning spills softly into your room, through the sleepy eyelids of your window shades. Your forehead is like a flip-book on which dreams unfold. Like that flip-book you made once, where a girl blows into a balloon, and the balloon is a heart, and it soars, and the girl is jumping with joy. And then an arrow comes and pops the balloon, and it spills water all over the place, and the girl is soaked and shivering, and the smile is wiped off her face. Because that's how things were in the past. And now you nestle your head against my shoulder and I hold my breath, afraid to disturb this perfect moment.

It's Sunday and we stay in bed till noon. You're taking seven classes, and I have a fucking thesis to write, but none of that matters right now, because you're here in my arms, and everything is as it should be. On the other side of the thin wall, your neighbor tries to drown us out with Christian music, but we just roll our eyes. Eventually our stomachs get the better of us, and we step barefoot into the kitchen. We fry some dumplings. I eat mine with soy sauce, and you laugh at me for avoiding the spicier stuff. You pop a yam into the oven, and I call it a potato just to get on your nerves. Let's play these games every Sunday; I wouldn't mind. I look at you and I feel warm inside. This feels like home. This is what I've been searching for all along.

Happiness is: Feeling like I belong. Feeling wanted.

Let's barricade ourselves on the couch and finish Funny Farm. Let's write a novel together. When December comes, let's make a snowman at Powder House Square. Then go to New York City for spring break. And in the summer, let's go read Alan Moore at the Fells. I'll teach you how to ride a bike. You'll teach me how to carve pumpkins for Halloween. Stay another day. Stay another year. Why did you have to leave so soon? I have so many things to tell you. I met Neal Stephenson and asked him about the antique-furniture scene. And did you know he uses a treadmill desk? You would've been so excited. You would've wanted to hear all about it. But now I'm here, in this weird place where it never thunders, and you are ashes in an urn somewhere. My memories keep swelling and spilling over, like a song on repeat, and I don't know which will end first: the rain, or my batteries.

If only it could all stop there,
As we walked back from Davis square,
Drinking that sweet November air
On Bristol Road.

Peter Thiel on Startups

2014-02-02T00:00:00-08:00

Blake Masters has made available notes from the course on startups that Peter Thiel taught at Stanford in the spring of 2012. These have been a lot of fun to read, and here are some bits I found particularly interesting:

We take for granted that competition is better than monopoly. But in perfect competition, no one makes any money, and all profits have to be reinvested. Every valuable business is a monopoly business, even though they pretend otherwise.
Many variables are distributed according to a power law, which is profoundly unintuitive, because we expect them to be roughly equal, not orders of magnitude apart. For example, out of 100 companies, one will make 100x profits, a few will make 10x profits, and the majority will hover near zero or even lose money. As another example, there is no reason to expect that multiple sources of energy (e.g. wind and solar) will have similar cost effectiveness; instead it is more likely that one will be vastly superior to all others.
When it comes to energy, conservation is not a viable solution, because demand in the developing world is growing faster than the gains from conservation in the developed world. Thiel points to thorium reactors as a potential solution.
There is no product so good that it "sells itself" without requiring a sales effort. Sales is omnipresent in tech, but we don't see it, because it works best when disguised. Really good salesmen appear to be just regular people.
We no longer believe that there are any frontiers remaining. That's why we focus on small, incremental improvements. But hard, discoverable secrets still exist, and in fact every successful company starts out by realizing something big that others don't realize. "What important truth do very few people agree with you on?"
Attitudes about luck and the future can be analyzed using two distinctions: determinate / indeterminate, and optimistic / pessimistic. Until the 1950s, people thought of luck as something to master and control. But today we think the future is unpredictable and out of our control.
People and companies default to having no plan. But that just means you're "floating", instead of moving in the direction you want. Companies should have a secret plan and execute on it. People should have a personal plan, too. "You should either like what you're doing, believe it's a direct plan to something else, or believe it's an indirect plan to something else."
The promise of biotech 2.0 is to understand our biology well enough to radically increase our life expectation. In other words, fix aging and death. (Opinion by Aubrey de Grey: The main roadblock to radical new technology is conservative public opinion. So mainstream opinion formers are going to be crucial for making progress.)

Self-Acceptance Without Resignation

2014-01-27T00:00:00-08:00

You probably have many flaws. I know I do! I've spent most of my life approaching these flaws in two ways: either I felt crappy and inferior for having them, or I worked my butt off to try and fix them. The first approach is painful and clearly doesn't lead anywhere. The second approach is self-delusional, because no matter how many flaws you "fix", you'll never get to the point where you can honestly say this:

When I first started reading about self-esteem and happiness, a third approach emerged: accept my flaws and live a good life in spite of them. The trick is to treat yourself with compassion, stop throwing insults at yourself in your head, and love yourself despite your flaws. (Cheesy, I know.) But there is something unsatisfying about this. Does accepting yourself mean that you give up on trying to become a better person?

I have a lot of faith in people's ability to change for the better, so I'm not comfortable with the idea that self-acceptance = resignation. I think we can illuminate this issue better by asking two questions: Does this flaw get in the way of your happiness? And: Can you do anything about it? Put the possible answers on two sides of a 2x2 matrix, and let's see what we've got.

	changeable	unchangeable
life-limiting	1	3
non-limiting	2	4

The life-limiting & changeable quadrant is where the action happens. Here are the flaws that make you unhappy, and that you believe you can change. Maybe you're overweight and you start an exercise program. Maybe your job is boring as hell, so you go look for a more satisfying one. Maybe you find it difficult to approach members of your preferred gender, so you get out and practice until it gets easier. (Fair warning: It's easy to feel overwhelmed if you're trying to change too many things at once. See the quadrant below.)

The non-limiting & changeable quadrant holds flaws that you can change, but that are not a high priority right now, because they don't interfere with your happiness as much as the ones in the quadrant above. Maybe you're terrified of heights, but you don't think it's worth your time to overcome this fear. Or maybe you wish you were a better speaker, but you decide to focus on other goals for now.

The life-limiting & unchangeable quadrant is the most painful. Maybe you have diabetes, and you hate having to watch what you eat. Or maybe you're extremely nervous around people, and you avoid social interactions. In both cases, the solution is to reframe the flaw and move it out of this quadrant. In the diabetes example, challenge the life-limiting aspect by focusing on all the fun things that you can do despite your chronic illness. The disease is a small part of you, and not the one major flaw that defines you. In the social anxiety example, challenge the unchangeable aspect by realizing that your problem is in fact treatable.

The non-limiting & unchangeable quadrant starts out empty, because if something really bothered you, it wouldn't be in this quadrant. This is where unchangeable flaws end up, if the reframing is successful.

Our updated matrix looks like this:

	changeable	unchangeable
life-limiting	CHANGE	REFRAME
non-limiting	DEFER	-

How does self-acceptance fit into this? In the CHANGE quadrant, you accept your flaw as temporary, and you give yourself credit for trying to change it. In the REFRAME quadrant, you accept your flaw, but focus on the things that make you happy despite it. In the DEFER quadrant, you again accept your flaw, but acknowledge that it's not worth fixing right now.

So you can have the best of both worlds: you accept yourself, and you also keep improving. Since the improvement process never ends, you have to make yourself happy on the way, and not think "I'll be happy once I fix this flaw."

California Road Trip

2014-01-18T00:00:00-08:00

This winter break my friend Raunak and I went on an incredible road trip to some of California's well known and lesser known treasures. We walked in deserts, hiked on mountains, camped on beaches, and had a few run-ins with law enforcement. Read on for the details! Photos are here. You should also check out Raunak's article about all the birds we've encountered.

Tuesday, December 24: It's Christmas Eve, Man!

On the first day we took the scenic Highway 1 almost all the way to Big Sur. We stopped at Moss Landing to check out the views, but as soon as we drove through the gate, an elderly old man -- shirtless and red in the face from drinking -- told us that he was going to close the gate in fifteen minutes. We inquired why they were closing so early at 4pm, to which he replied: "It's Christmas Eve, man!"

We took a detour to Elkhorn Slough to watch the sunset. Being too cheap for motels on this trip, we were thinking of spending the night there. But unfortunately they had signs that said "no overnight parking". So we drove south to Carmel, climbed a steep residential road, parked along someone's fence, and hoped we wouldn't be disturbed. We waited an hour or so to make sure no one would come, and then tried to sleep. It was a long night, from about 7:30 pm till 6 am, and I don't think we slept at all for the first half. (At one point someone came and looked at us with a flashlight, but they didn't say anything. Or maybe I dreamed the whole thing.)

Wednesday, December 25: Pushing Our Luck

In the morning we drove down to the nearby Point Lobos State Reserve. Thinking that it would be closed on Christmas, we parked near the entrance, jumped the gate, and went to explore. The views were gorgeous, and the sound and smell of the ocean made them even better.

When we got back to the gate, we saw that the park had opened after all. We walked to our car, trying to look innocent, and nobody said anything. So we high-fived each other and drove off. It was only about fifteen miles later that we noticed the ticket on our windshield. $143 for "parking off road" and "non-payment of park fees"...

Our next destination was Big Sur. Before we got there we stopped at a gas station, squinted our eyes at the ridiculous prices, and stopped to take pictures of stellar jays, to the bafflement of the gas station attendant. (There was also a brief but memorable moment of driving on the wrong side of the road.) At Big Sur, we paid the hefty $35 for a campground, left our car, and went to hike the Pine Ridge trail. The hike was rather underwhelming, since the trail was too long to get anywhere within a few hours, but at least it was mostly in shade.

Back at the campground we found that the squirrels had air-bombed our car, which was now littered with nuts. We set up our tent and went to bed.

Thursday, December 26: Miscellaneous

The night was pretty warm, although the wind made our tent behave like a sail, and I kept dreaming I was on a ship. (Come to think of it, I've never slept on a ship. Sounds fun.) Raunak also heard raccoons at night. In the morning we did a short hike to a small waterfall, and then to a point where you could see the ocean.

Continuing to drive south we passed through Harmony, CA, which has an amusingly detailed Wikipedia page for its total population of 18. We also saw a beach swarming with elephant seals. When we finally stopped to get gas, we were amazed that our mighty little Yaris moved us 250 miles on only 8 gallons.

We stopped to see the beach at Montaña De Oro, with its magnificent dunes of clean white sand. We saw something on one of the dunes, which Raunak thought was a seagull, and I thought was a piece of trash. Then the piece of trash raised its head and revealed itself to be a woman with a hat. She couldn't figure out why we were laughing so hard.

We stayed on the beach to watch the sunset, drove to Morro Bay for dinner, and then took off for our next destination: Carrizo Plain. That night we slept in the car, on a dirt road between two hills. The sky was bright enough to see the Milky Way.

Friday, December 27: Walking on the Moon

It was 3°C (38°F) when we got up. As we drove downhill trying to get a closer look at Soda Lake, the temperature dropped to -8°C (17°F). Eventually we got to a dirt road that was too much for our little Yaris. We put on all our extra layers and decided to walk the rest of the way. Despite the early-morning sun and the vigorous walking, our fingers and ears nearly fell off from the cold.

This was by far the most surreal part of our trip. There was no one else as far as the eye could see, and if we stopped moving, the silence was all-enveloping. (You never get real silence in a city, and even if you go to the forest, there is the sound of birds chirping or leaves rustling. Here, there was nothing.) The earth under our feet was soft and gray, and our shoes sank an inch into it with every footstep. It was like walking on the Moon.

The pools of water around us were covered with a hard, white substance that crunched when we stepped on it. At first we thought it was ice, but upon further reflection we realized it was salt. All the water had dried, and only the salt remained. When we got to Soda Lake it was the same -- a large expanse of dry, white salt. Even on Google Maps, it looks otherworldly.

(Later we found the park's visitor center and learned some more details. Apparently, after the rainy season, the lake fills with water and the entire plain is covered with flowers. But we didn't get to see this, because there was no rain this year.)

After a few short hikes, we started driving towards our next stop, Mount Pinos. The road traversing Carrizo Plain turned out to be mostly unpaved, but despite this lack of development, we saw signs about a trans-continental AT&T cable. Who knows why they decided to route it through the middle of nowhere.

After a small incident involving stopping on the highway to photograph a bird, we did a short hike in the Mount Pinos area, where parts of the trail were covered with snow. It was already past sunset. We decided we'd drive to Castaic Lake and find a place to sleep there. Google Maps led us straight into a power plant, where an armed but friendly guard informed us that we'd taken a wrong turn. We turned around and drove a few miles, stopped at a turnout, and slept in the car.

Saturday, December 28: Blame Everything on L.A.

Castaic Lake was gorgeous in the morning. Boaters were lining up as early as 6:45 am. We went down to the lake, met some fishermen, and then found a trail so steep that it required a rope (thoughtfully left there by previous adventurers).

Next, we briefly stopped at a McDonald's to shave (oh, the luxury) and to use the bathroom. After that we found our way to CA 23, a very bendy mountain road popular with cyclists, which took us all the way to Malibu. The beach and the water there were beautiful.

Across the road from the beach was a string of expensive-looking houses. On one end of the beach, there was a cliff where people rock-climbed.

Our next destination was Griffith Park, which had been recommended to us by someone we met that morning. Getting there proved to be quite an adventure. We circled around half of L.A. because we couldn't manage to get off the freeway, since as soon as we took an exit, it somehow brought us back onto another freeway. After so much time gloriously away from civilization, we were not thrilled to find ourselves in a traffic jam at Griffith Park, nor in the swarm of people photographing the Hollywood sign. It was then we decided that whatever bad things happened on our trip, it would all be L.A.'s fault. We had no idea that our troubles were just beginning.

We needed an ATM, so we decided to go see downtown. We parked our car at a multi-level garage. (It was probably then that my tablet got stolen from the passenger seat pocket -- a fact that did not become apparent until the end of our trip. Argh.) Downtown L.A. looked to me just like Manhattan, and just as dispiriting. We quickly discovered that GPS didn't work so well among skyscrapers, and when we finally found the ATM, it was at the opposite end of the block from where Google Maps thought it was.

We met up with some of Raunak's labmates for dinner. Then we drove to Laguna Coast Wilderness Park, hoping to find a place to sleep. But everything there was behind closed gates. We were still too close to L.A., and under its bad karma. So we drove to Palomar Mountain State Park, stopped on a turnout, and slept in the car again. At this point it was past midnight -- one of our longest days driving.

Sunday, December 29: Dodging the Bullet

At Palomar Mountain we had one of the best hikes of our entire trip. I couldn't get enough of the changing scenery -- pond, oak forest, grass clearing, evergreens.

We also found a fire lookout tower with a long vertical ladder. There was a fence around it, but the gate was unlocked, and there were no "no trespassing" signs. So, naturally, we decided to climb it. I went up first. One minute into my climb I heard noises below. I looked down and sure enough, a fire truck and a park ranger car were coming up the road. I quickly got off the ladder and onto one of the tower's platforms, then ducked so they hopefully wouldn't see me. I was already rehearsing what I would say if caught. ("Fire tower? What fire tower? I thought this was an observation deck! No signs down below, so how was I supposed to know?")

Luckily, they didn't come up to the tower to look. They just waited a few minutes, then left. I sat up for just a second to enjoy the amazing, unobstructed 360° view. Then I climbed down quickly, knowing that I was also very visible from 360°. Later Raunak told me that the fire people looked and looked, but that when I sat on the platform, I was invisible from below. Good thing I don't wear bright colors.

Just to be safe, we took an alternate road out of the park, which turned out to be narrow and unpaved for most of the way. But the views were nice, and we saw a lot of lemon trees. (And casinos. In Indian reservations. The peculiarities of US law never cease to amaze me.)

While driving to San Diego, I saw two things for the first time: a 70 MPH speed limit, and a 9-lane freeway. We booked a motel for the night (with some resistance from the lady on the phone, who wanted to add me to some $19/month "advantage" program, and who wouldn't accept no for an answer until I said it three times). We then drove down to the Border Field State Park, threw a stone over the fence to Mexico, came back to the motel, took our first showers in 7 days, and then slept for 10 hours straight.

Monday, December 30: Night Hike to Oasis

We got up at 6, picked up Raunak's birding pal, and spent the first half of the day watching birds at the San Diego River. Then we restocked on food (bread, cheese, and cold cuts mostly), and got on the road towards Anza-Borrego. The road through the mountains was bendy and made our heads spin. We got to Borrego Springs about half an hour before sunset, and promptly decided to do a short hike while there was still light.

At the beginning it was really dry, with rocks, sand, cacti and brownish desert shrubs all around us. Then we saw trees and shrubs that were actually green. Then we heard the sound of water, and soon enough we found a small stream. We followed it up and found an grove of thick-bearded palm trees. (This was definitely not a mirage.)

The way back was interesting. It was already dark, and we had a flashlight. But the narrow cone of light that the flashlight created meant that we could see very little around us, and so we lost the trail multiple times before finally getting back to our car. We drove to the east side of Anza-Borrego and set up our tent on a turnout, grateful that this park, at least, lets people camp anywhere without a permit.

Tuesday, December 31: Popcorn Beaches

The first order of the day was to do the Palm Slot trail. Of course, we missed the trailhead and had to walk an extra mile each way, almost giving up before we found it. But it was absolutely worth it! The trail got narrower and narrower, until we were walking between the walls of a deep, narrow canyon.

We walked the long walk back to our car, ate some brunch, and headed towards our next destination: Salton Sea. While still in the desert, we saw a lot of people having fun with off-road cars and motorcycles. (Note for next time: Bring a dirt bike.)

Salton Sea has an odd history: it's a man-made lake created by accident, at an elevation of 69 meters (226 feet) below sea level. The beaches are made of a thick layer of barnacles, instead of sand. Imagine walking on miles and miles of popcorn.

This sea is already saltier than the ocean, and it's getting saltier as time passes. Despite the increased salinity, the sea is still teeming with fish, and it is also a haven for birds. We spent a long time watching the hundreds of geese, gulls, pelicans, and so on. There was a railroad nearby, and we saw quite a number of freight trains pass by. The longest had 117 cars. Living in the city, it's easy to forget that freight railroads are still a huge part of the transportation economy.

And so we watched the last sunset of 2013 on the shore of Salton Sea. We then ate some of Trader Joe's carrot cake to celebrate the new year, set up our tent on the beach, and got into our sleeping bags. There were maybe five other people there altogether. I could not imagine a more perfect end to the year.

Wednesday, January 1: The End / The Beginning

The slight wind at night made our tent rustle, and I kept thinking I heard footsteps. But the night passed uneventfully, and we got up at 6 am to drive a bit further south and catch the sunrise.

We visited a few more places on the south shore of Salton Sea, marveling again at the number of birds, and also at the number of geothermal plants around the area. We found a part of the beach that was full of obsidian. Nearby in the water was a dead tree with at least seven nests on it.

We then drove to Calexico, a small town bordering the much larger Mexicali across the border. We saw the fence that runs between the cities, and the green-and-white border patrol cars swarming all over the place.

When it was time to drive back to San Diego, we took highway 98 and then 94, which promised a more enjoyable drive than the crowded Interstate 8. We were rewarded with a succession of unusual scenery: flocks of sheep, then large installations of solar panels, then wind turbines, and finally some enormous hills made of small rocks. We passed through a border patrol checkpoint, where an unsmiling officer reminded us that we must carry our passports at all times, or else they might revoke our student status.

Finally, eight days and 1,100 miles later, Raunak dropped me off in San Diego, where I wanted to stay for a few days. He then drove all the way back to Berkeley in one heroic, 9-hour, 500-mile stretch of Interstate 5. This was the end of our road trip, but we felt that we've started 2014 on the right foot. Who knows what other adventures the year will bring?

Praising NetworkManager

2013-12-10T00:00:00-08:00

Networking has gotten a lot easier in GNU/Linux in recent years. Two experiences made me really appreciate this:

1) My laptop has built-in wifi, but I was testing a USB wifi adapter. When I plugged in the latter, Ubuntu's NetworkManager applet allowed me to connect each adapter to a different network. The GUI is smart enough to handle more than one wifi adapter!

2) My laptop connects to the Internet via wifi. I connected a BeagleBone embedded computer via ethernet, and wanted to set up my laptop as a gateway, so that the BeagleBone could connect to the Internet via my laptop. I was prepared to have to set up dhcpd and dnsmasq and so on. But all I had to do was tell NetworkManager that I wanted to share my connection, and it set up everything for me!

Togetherness

2013-11-29T00:00:00-08:00

Friends walking on water at Ocean Beach

Reach

2013-11-18T00:00:00-08:00

Looking for angels on Angel Island.

Zen and the Art of Motorcycle Maintenance

2013-11-07T00:00:00-08:00

On two ways to climb a mountain (p266-268):

When you try to climb a mountain to prove how big you are, you almost never make it. And even if you do it's a hollow victory. In order to sustain the victory you have to prove yourself again and again, driven forever to fill a false image, haunted by the fear that the image is not true and someone will find out. That's never the way.

[...]

To the untrained eye ego-climbing and selfless climbing may appear identical. Both kinds of climbers place one foot in front of the other. Both breathe in and out at the same rate. Both stop when tired. Both go forward when rested. But what a difference! The ego-climber is like an instrument that's out of adjustment. He puts his foot down an instant too soon or too late. He's likely to miss a beautiful passage of sunlight through the trees. He goes on when the sloppiness of his step shows he's tired. He rests at odd times. He looks up the trail trying to see what's ahead even when he knows what's ahead because he just looked a second before. He goes too fast or too slow for the conditions and when he talks his talk is forever about somewhere else, something else. He's here but he's not here. He rejects the here, is unhappy with it, wants to be farther up the trail but when he gets there will be just as unhappy because the it will be "here." What he's looking for, what he wants, is all around him, but he doesn't want that because it is all around him. Every step's an effort, both physically and spiritually, because he imagines his goal to be external and distant.

More of the same (p418):

I watch the cars go by for a while on the highway. Something lonely about them. [...] Something about the car drivers too. [...] They all look like they're in a funeral procession. [...] The driving is different too. The cars seem to be moving at a steady maximum speed for in-town driving, as though they want to get somewhere, as though what's here right now is just something to get through. The drivers seem to be thinking about where they want to be rather than where they are. [...] Folks, I just forgot the biggest gumption trap of all. The funeral procession! The one everybody's in, this hyped-up, fuck-you, supermodern, ego style of life that thinks it owns this country.

From the book by Robert M. Pirsig (emphases mine).

Hosting Multiple Domains on a VirtualBox VM

2013-10-23T00:00:00-07:00

Let's say you have a production server configured to serve two domains: thirld.com and example.com. For testing, you also have a local virtual machine serving these domains. How can you visit these domains as served by your local VM?

The first step is to make sure your host machine can talk to the VM. By default, VirtualBox VMs are set up with NAT networking, which means that the VM can access the Internet, but the host can't access the VM, since it has no way to address it. To fix this:

Go to VirtualBox settings (the global ones, not the VM's settings), and create a host-only network vboxnet0. Make sure it has DHCP enabled.
Go to your VM's settings and add vboxnet0 as a second, host-only network adapter.
In your VM, add the following to /etc/network/interfaces:
```
auto eth1
iface eth1 inet dhcp
```
Then start the interface with ifup eth1. The VM will get an IP address like 192.168.56.101, while the host has an IP address like 192.168.56.1. Use ifconfig to figure these out.

At this point, you should verify that the host can ping the VM, and the VM can still access the Internet. (This is why we kept the first network adapter as NAT; otherwise the VM couldn't access the Internet.)

The final step is to fool your host into believing that your domains actually live on the VM. On the host machine, add the following to /etc/hosts:

192.168.56.101 thirld.com example.com

Now load those domains in your browser. They will hit the VM instead of the production server. Congrats, you have a way to test your websites locally before pushing them to production. The main benefit of this approach is that your VM and your production server can have exactly the same configuration, so you don't need to keep separate configs for production and testing.

(Note that if you loaded 192.168.56.101 in your browser directly, you'd only be able to see one of the domains, and not the other. Also, if your website uses absolute URLs, various resources such as CSS files would be loaded from the production server rather than your VM. This is why the /etc/hosts trick is necessary.)

Graduation

2013-09-25T00:00:00-07:00

Alone in your room --
the walls are bare;
the drawers are empty.
In the middle of the floor,
two suitcases stare at each other
counting,
in disbelief,
the cubic centimeters of your life.

You put Yann Tiersen in your ears and pretend it's just another day.

Red line.
Silver line.
Airport.
To fit under the weight limit,
you take eight pounds of clothes
and feed them to the gaping mouth
of a garbage bin.

Past security and into no man's land.
The plane is late four hours.
Your grip on the world feels thin.
You hug your backpack and try not to disappear
like a helium balloon
lifted by the absence of keys in your pockets.

But still you're not entirely your own.
Memories pull at you like strings,
and they will have to stretch across
three more time zones now.

Home is where what is?

(I wrote a first version of this in May 2012, but I wasn't happy with it. Every few months, I'd try to edit it, get frustrated, and give up. This is just the latest iteration.)

How to Get Vi / Vim Keybindings in Bash, IPython, etc.

2013-09-21T00:00:00-07:00

A lot of programs come with emacs keybindings by default. If you're a vim user, you can make your life easier and more productive by switching to vi mode instead. This lets you edit your command line using familiar vi shortcuts (ESC to enter command mode, B to go one word back, dt/ to delete until the next slash, and so on).

To get vi keybindings in bash, add the following to your ~/.bashrc:

set -o vi
bind -m vi-insert "\C-l":clear-screen

To get vi keybindings in IPython, the MySQL client, and many other tools that use the GNU Readline library, add the following to your ~/.inputrc:

set editing-mode vi
C-L: clear-screen
C-l: clear-screen

(The C-l thing is a useful shortcut for clearing the screen. It's there by default in emacs mode, but in vi mode it has to be specified by hand.)

Mount Tallac at Lake Tahoe

2013-09-18T00:00:00-07:00

Probably the most scenic hike I've been on so far.

(more pictures)

Visiting Paris

2013-08-28T00:00:00-07:00

With the noise of so many people and the weight of so much history, I don't know how the locals find peace in this city...

(more pictures)

Candy clouds

2013-08-15T00:00:00-07:00

Near Hîrjauca, Moldova.

(more pictures)

Visiting Copenhagen

2013-07-20T00:00:00-07:00

Denmark in summer = cyclist's paradise. Bike lanes everywhere, elevated from street level, and wide enough for two people to ride side by side. Also, you should see the locks they use. Somehow bike theft isn't nearly as prevalent as in the US.

(more pictures)

A Head-Scratcher

2013-07-07T00:00:00-07:00

I read this and then I had to think for a very long time.

Right now you believe that your worth depends on your behavior. Metaphorically, you see yourself as an empty vessel that must be filled, drop by drop, with your achievements. You start out essentially worthless, a body that moves and talks. [... There] is no intrinsic value in life, only a potential for doing something worthwhile, something important.

The truth is that your value is your consciousness, your ability to perceive and experience. The value of a human life is that it exists. [...] You are a person who is trying to live, and that makes you as worthwhile as every other person who is doing the very same thing. Achievement has nothing to do with it. Whatever you do, whatever you contribute should come not from the need to prove your value, but from the natural flow of your aliveness. What you do should come from the drive to fully live, rather than the fight to justify yourself.

From the book Self Esteem by Matthew McKay and Patrick Fanning.

Moxie Marlinspike's Stories

2013-06-09T00:00:00-07:00

I first learned about Moxie Marlinspike when I found TextSecure, his open-source app for encrypted text messages. More recently I stumbled upon his website, and there I found some amazing stories about sneaking into movie theatres, sailing around the world with no engine, and hitchhiking / riding freight trains across America. I didn't think that such adventures were still possible in today's highly regimented world.

At times I, too, have longed to show civilization the finger, to go live somewhere far away and be autonomous and self-sufficient. Unfortunately, these wishes have only manifested themselves in a few angry, solitary bike rides off into the wilderness. Every time I contemplate more interesting adventures (like jumping a fence to where I'm not supposed to be), I remind myself that I am here on a visa, and that even a slight misdemeanor could be sufficient excuse to kick me out of the US and never let me back in again.

Moxie, on the other hand, has no qualms about confronting authority and going his own way, even if he gets burned once in a while. I really admire that.

A Weekend in Portland

2013-06-04T00:00:00-07:00

Somehow the colors seemed more intense in Portland. The sky was a deeper blue, and the vegetation was a more vivid green. Later I saw walnut and maple and lilac trees, just like back home. And I realized I missed them in California.

(more pictures)

South San Francisco

2013-06-02T00:00:00-07:00

How I will remember you:

The fog. The junk mail. The bottle-brush bushes.
The whiteboard that only lied flat on cold days.
The crows that woke me up in the morning
(and made me wish that I had a gun).

The old guard who waved me in with a grin
every night, as I passed through the gate.
Then the new guard who didn't smile,
and who wanted to see my ID.

The homeless woman wrapped in garbage bags,
talking to herself in a high-pitched voice,
moving a shopping cart from here
to there, and then back again.

How the yellow hills turned green only in winter.
How it was never warm enough to sit down.
How the bike trail cowered and crawled
under the tentacles of the highway.

Long minutes waiting for pedestrian lights.
Riding my bike against 30-mph wind.
Coming home to a queen-size bed
and nobody to share it with me.

Uncomfortable red-eye flights to Long Island.
Dreaming about people I didn't get to know.
Moving on to the next temporary place
in a long string of temporary places.

A Fork Holding the Road

2013-05-28T00:00:00-07:00

Freeway 280 passing over Crystal Springs Road.

Joel Barish on the Bus

2013-05-21T00:00:00-07:00

The sun has already set when you leave the office, and the world seems no longer capable of colors. Another short December day, and you'll be thirty-eight. You hide your hands deep inside your pockets, knowing that the cold will find them anyway. In the parking lot, your car is not in its usual place, and you look around panicked for a moment. Then you remember that it won't be back from its annual checkup until Thursday. Resigned, you turn and head for the bus stop.

Under your feet, the pavement is soggy and soiled. The crows keep their heads low and fluff their feathers, trying to stay warm as they probe the cracks for anything edible. An old lady buries her chin in a scarf, her nostrils releasing a puff of steam. There is no one else at the bus stop. The lady is wearing gray. The crows are wearing black. You're wearing black. The old lady looks at you with disapproval. The crows look at you with disapproval. You look at your feet. The bus arrives. You find an empty seat near the back, and lean your head against the window. You close your eyes, and let your bones resonate with the motor's monotonous rumble.

It's April. You're sixteen. A clear orange sun shines above verdant lindens. The asphalt sparkles and shimmers with the heat. At last, the school bus arrives, and you climb aboard. Inside, it smells like too much air freshener. Erica is sitting near the front, and you remind yourself to breathe as you take in her unclouded, perfectly sculpted face. Her hands are playing absentmindedly with the straps of her backpack, which is resting on her lap. You hesitate. There's an empty seat next to her. Your throat is dry and your hands start to sweat. If only you knew what to say. Every time you see her, this happens. And every time, you feel like a worthless piece of shit. You wish that she didn't affect you in this way.

So you keep walking, and sit down across the aisle, a few rows behind her. From here you watch as her delicate fingers rise and comb through her straight black hair. You imagine what her hand would feel like if she did that to you instead. She's wearing her tiny green shoes again, and you notice that her right heel is out of its shoe and resting on the hem. That makes you feel ashamed, for some reason. You look away and think of the other times you've watched her. How she sat in the cafeteria with her back straight, when everyone else was slouching. How she kept her chin up when a senior poked a dirty jibe at her in the hallway. How she volunteered to decorate the homeroom for Christmas.

At the next stop, your friend Charlie gets on the bus. You give him a small wave, but he doesn't see you. He goes straight for the empty seat next to Erica. Charlie, who can pull a joke out of thin air, and who always beats you at chess. He has no trouble starting a conversation with her. You watch with powerless envy as Erica tilts her head to one side and laughs. And you tell yourself that it's okay, that in a few years it will all make sense, and this will be just as easy for you. As the bus moves, the sunlight filters through the trees in ragged bursts. You close your eyes and let the light fall on your face, in a violent dance of orange and red flashes.

"Hey buddy, last stop," the driver wakes you with a firm tap on the shoulder. It's dark outside. You look around and the bus is empty. Everything looks dirty and gray under the electric lights. You mumble a vague apology and step off. Your feet are numb. You pull your hat further down to cover your ears. Walking to your apartment, you can't stop feeling like you forgot something important, but you just can't remember what it was.

New Static Blog Is Finally Up

2013-05-17T00:00:00-07:00

At some point in the last 3 years, I discovered Git and fell in love. While mostly used for code, Git is a fantastic tool when editing text files of any kind. My favorite part is that I can see exactly what I changed (diff) before saving my changes (commit). At this point, I feel uncomfortable editing anything, unless it's under version control.

At some point in the past year, I discovered Markdown and fell in love all over again. If you've ever posted a question or answer on Stack Overflow, you know what Markdown is. You type plain text with minimal hints (## heading, _italic_, **bold**), and it gives you back HTML.

file format	source is readable	output is beautiful
plain text	✔
HTML		✔
Markdown	✔	✔

Once I started keeping my notes in Markdown and Git, I knew I could never go back to the quirky and unreliable WYSIWYG editor that WordPress used. Luckily, other people have come to this realization before me, so now there are plenty of tools that let you write your blog in Markdown and store it in Git. These tools are called static site generators. They take the source for your articles (written in Markdown or a handful of similar formats), and convert it into a set of static HTML files. You upload those files to your server, and you're done! It's faster than WordPress, since you're just serving static HTML. It's safer, too, because you don't need to keep up with security patches in WordPress and PHP.

I first tried Jekyll, since that's the first static site generator I heard about. The idea was great, but I found Jekyll itself unintuitive, poorly documented, and difficult to extend (partly because I don't know Ruby). Then I tried Pelican, and that is what I'm using now. Pelican is written in Python and has generous documentation. I am very happy with it so far.

The one big downside of static sites is that they don't support comments. You need a database and some server-side magic for that. To work around this problem, a lot of people turn to Disqus, or a similar "cloud"-based commenting component. I don't like Disqus because it's not open source, and it performs user tracking across sites. But I love reading comments, so I decided to include an email link in each article. This means that your comments will not show up on the site, but only in my inbox. Most of the comments I got in the past were directed at me, rather than at other commenters, so I hope this will work out fine.

I've been working on this new blog on and off for almost nine months. Coming up with a theme that I liked took the longest time, and I almost gave up more than once. (CSS frustrates me to no end.) Eventually I decided to do something really simple with Bootstrap, and call it done. I sort of like the minimalist style. I made the font size large, because I usually find myself zooming in on most sites. If you think it's too big, feel free to use Ctrl+Minus to scale it down. Any decent browser will remember your preference.

I've "published" a few things in those nine months, that only became visible now. Mostly thoughts on books and photos, but also a few xkcd-wannabe sketches. You can see the entire list of articles on the front page. Thanks for stopping by :)

The Monster Awakens

2013-05-07T00:00:00-07:00

The Vaillancourt fountain at the Embarcadero.
This is why I really need to get a tripod.

(more pictures taken for dpchallenge.com)

A Taste of Taipei

2013-05-06T00:00:00-07:00

The Chinese Pavilion in Golden Gate Park is a gift from Taipei, San Francisco's sister city.

(more pictures)

Damselfly

2013-05-04T00:00:00-07:00

Today at the Berkeley Rose Garden.
She was anything but shy.

(more pictures)

Disappointment

2013-05-04T00:00:00-07:00

I wrote a poem for you yesterday,
Because my heart pumped hope instead of blood,
Because I saw your wings and fell in love
With their exquisite twists of blue and gray.

You beckoned me to play amid the noise
Of water falling from a jagged cliff;
I chased you through the spray and through the mist,
Enchanted by the laughter in your voice

As night consumed the sun's remaining rays,
A nervous knot took shelter in my throat.
You pulled me to your bower, and I thought
I would, at last, find peace in your embrace.

But when I was about to bare myself
To you, I noticed the transparent strings
That pulled ever so subtly at your wings,
Enslaving you, like everybody else.

I'll write a poem for myself tonight;
The puppets will forget their strings and dance,
And I'll pretend that I was not entranced
By yet another paper butterfly.

(I wanted to edit this and make it more coherent, but that process got stuck in an infinite loop. Now too much time has passed, so I'm just backdating this to the time I wrote it, and publishing it as unpolished as it is.)

A Conversation between Julian Assange and Eric Schmidt

2013-04-20T00:00:00-07:00

Julian Assange is the man behind WikiLeaks. Eric Schmidt is the CEO of Google. They met secretly in 2011, and now you can read a transcript of their conversation.

(This is not as illicit as it seems. Schmidt supposedly interviewed Assange for his book, The New Digital Age. The transcript was released online on April 19, 2013, four days before the scheduled publication of the book. I'm pretty sure they coordinated the release of the transcript, to generate interest in the book.)

The transcript is pretty long, but well worth reading. Here are some points that I found especially intriguing:

Addressing content by its hash, rather than by a name. Names (e.g. domain names) are scarce, and they require a hierarchical structure, which can be abused by those who control it. Content stored under a name can be censored or removed, and such alterations leave no trace. Furthermore, you can only get the content from the owner of the name. In contrast, if you change the content, the hash changes, and it is computationally difficult to forge content that maps to the same hash. Moreover, you can get the content from any node that has a copy, so it becomes more difficult to make content disappear. Peer-to-peer file sharing networks already have some technologies for addressing content by its hash, such as magnet links and distributed hash tables.
Preserving an un-forgeable, un-eraseable history of all content, again using cryptographic hashes. Git and Bitcoin are some technologies that do this already.
The key infrastructure is often the weakest link in a crypto system. Steal a key, and you can decrypt tons of content, and forge content too.
Encrypted peer-to-peer networks made of mobile phones. If your government decides to shut down the big ISPs, you can still communicate.
The young generation getting their political education from the Internet.
In China, the ruling powers are precarious. They have to worry about what people think, hence censorship. In the US, the ruling powers are strong. They don't care what the public thinks, hence free speech.
How censorship works: People censor what their boss will see, to cover their own asses. They're not that concerned about information leaking onto darknets.
Avoiding scrutiny through complexity: Make something so complex, that it is too difficult for your opponents to understand it. Example: evading taxes by building elaborate financial structures.
How to distinguish truthful publishers from liars? Require citations to primary sources? Devise some system for verifying someone's reputation? Will we see AI systems designed to produce vast amounts of misinformation, drowning out the truth?
Using encrypted email is like raising a red flag, saying "I'm doing something I don't want you to know about. Come install a rootkit on my machine." This wouldn't be the case if more people used encrypted email.

Power on the Internet is becoming increasingly centralized. For example, NAT boxes limit end-node connectivity, and proponents of the "cloud" want you to rely on a few big and powerful players. This makes it easy to be a consumer of content, and hard to be a producer (especially if your content is controversial). Reading Assange and Schmidt's conversation has reignited my excitement about peer-to-peer networks, where everyone is an equal, and you don't have to trust or depend on a central authority. Sounds like something from a Charles Stross novel. I think the next 50 years are going to be very interesting.

Nuclear Perspective

2013-04-03T00:00:00-07:00

No Spoon

2013-03-31T00:00:00-07:00

Semi-nyms

2013-03-29T00:00:00-07:00

Janis Ian - At Seventeen

2013-03-09T00:00:00-08:00

I love it when authors tell how they got the idea for a story. For example, Orson Scott Card's Inventing Lovers on the Phone was inspired by this song. Cheesy but I like it :) What singer today would use words like 'debenture'?

Helvetica

2013-03-07T00:00:00-08:00

A little stop-motion animation I made :)

A Binder Clip with Wings

2013-03-05T00:00:00-08:00

My first ever dpchallenge entry that finished in the top 10!

Beyond Fear

2013-03-01T00:00:00-08:00

Bruce Schneier's Beyond Fear is an excellent introduction to thinking rationally about security. The book is packed with insights, but still very readable. And it uses real-world examples to illustrate the concepts being discussed. This post is a summary of the notes I took while reading the book.

"Fear is the barrier between ignorance and understanding," says Schneier in the first chapter. To think rationally about security, you have to be dispassionate. You have to look at statistics, instead of watching the news and thinking that it will happen to you tomorrow (see availability heuristic). Real risk = threat * likelihood of successful attack. Perfect security is unachievable. (Want to have zero airplane hijackings? Ban all commercial flights.) Schneier proposes a five-step process for evaluating a proposed security measure:

What assets are you trying to protect?
What are the risks to those assets? Here you have to understand your attackers -- know their motivations, goals, expertise and resources, level of access (are they insiders?), and risk aversion.
How well does the security measure mitigate those risks? Here you have to understand the players involved -- know their agenda and power to influence. Are they selling real security or just security theater? Are they just trying to move the blame somewhere else in case something bad happens? To get a player to mitigate a risk, make him or her accountable for it / liable for the damages. (Conflicts of interest abound. Why can you take matches on a plane? Because the tobacco industry lobbied to allow it. This is just one example of a security decision made for business, not security reasons.)
What other risks does the security measure cause? Does it give a player too much power? Does it move the weakest link somewhere else, leaving the system just as vulnerable? Does it cause so many false alarms that people will simply ignore it, like in The Boy Who Cried Wolf?
What tradeoffs does the security measure require? Does it cost money? Does it destroy privacy and dignity? Does it make the system so difficult to use that people will turn elsewhere?

All security is about tradeoffs. Do you lock your door while you walk downstairs to the laundromat (more secure), or do you leave it unlocked (more convenient)? Ultimately, it is a subjective decision whether a security measure is worth its tradeoffs. The five-step process above is meant to help you consider all the factors involved when making such decisions.

The sole purpose of security systems is to prevent some things from being done. Security is invisible when it works. That's why we have to think about how systems fail. While a system may offer good protection against "frontal" attacks, most security breaches happen at the seams between secure systems (during a shift change, while the money is being carried from the armored car to the vault, etc). Complex, tightly coupled systems with many possible interactions are harder to secure. A security system can fail in two ways: passive failures (the system fails to react to an attack) and active failures (the system raises an alarm when there's no attack).

A system can be brittle (a small failure leads to complete compromise) or resilient (flexible, able to recover from failures). Static systems (cannot adapt), homogeneous systems (vulnerable to class breaks; see below), and systems that rely on secrecy (security through obscurity) tend to be brittle. It is tempting to think that an ideal security system would be fully automated, with no risk of human error or malice. But technology today is static and brittle, while people are adaptable and resilient. (A person can feel suspicious and investigate further; a computer can't.) So secure systems require trusted people, and probably will do so for the foreseeable future. (Side note: People make mistakes, so social engineering will probably always work.)

Computers and the Internet give attackers some leverage. Once you figure out how to break one system, you can break many other systems of the same type. Schneier calls this a class break; others use the term monoculture. (Side note: Nature protects against class breaks through diversity, but human agriculture, industry, and software tends towards homogeneity. Diversity sacrifices the individual's security for the security of the overall population -- an idea that doesn't sit well when the individuals in question are people.) Thanks to data aggregation, attacking a single system can have a huge payoff (steal one million credit cards). Through automation, attacks that have a low payoff (steal one cent from a million people) or a low probability of success (click on a spam link) suddenly become profitable. And the Internet enables attackers to act at a distance, taking advantage of the friction between legal jurisdictions. But it is important to remember that technology is inherently neutral, and that any technology can be used for good or for evil. By making a technology available, we (as a society) decide that its benefits outweigh the potential harm.

There are a few timeless strategies that make for good security. Defense in depth means having multiple barriers, so that an attacker has to penetrate all of them to break the system. (Example: a medieval castle.) Compartmentalization means splitting the system into chunks that are secure by themselves, and a successful attack on one chunk does not bring down the entire system. (Example: storing money in more than one place when traveling.) A choke point funnels all players into a narrow space, where security measures are concentrated. (Examples: the security checkpoint at an airport; the strait of Gibraltar). In general, a tried-and-true security system is better than a new and untested one.

Secure systems have to let the good guys in, while keeping the bad guys out. This is done by the triangle of identification (Who are you?), authentication (Prove it.) and authorization (Here is what you are allowed to do.) The authentication step can check something you know (a password), something you have (a physical key), something you are (a fingerprint scan), or any combination of the above. Conflating identification and authentication leads to poor security. For example, a social security number in the US is an identification token -- it's not secret, and it can't be changed. Using it for authentication is a bad idea. As another example, biometric signatures are good for authentication (Does the face in front of the camera correspond to person X?), but bad for identification (Which of these one million persons does this face correspond to? -- a much harder question). Two final curiosities: Do you know what an FBI badge looks like? An authentication system isn't secure if it's unfamiliar. Did you ever wonder why airlines required photo ID, long before 9/11? It wasn't for security -- they just wanted to prevent people from reselling tickets.

Prevention is often expensive and hard to get right. In many cases, detection during or after an attack is cheaper and more effective. For example, most surveillance cameras are not being watched live. Their role is to generate an audit trail that can be checked later, in case some security incident has occurred. Detection would be useless without an appropriate response, which completes the triangle. There are many types of responses. For example, forensics (trying to figure out what happened and how) is at odds with recovery (returning the system to a normal state), because recovery often destroys evidence.

A mandatory paragraph on terrorism. The main target of terrorists is morale -- they want us to live in fear. So wide publication of terrorist attacks in the media actually furthers the terrorists' goals. The measures put in place by governments to "prevent terrorism" are largely ineffective, because attacks are very rare, the target is everything, and perfect prevention is impossible. On the other hand, these measures cost dearly in terms of dollars and civil liberties. A much more sane approach would be to say, "Terrorist attacks are extremely rare, but they will happen from time to time no matter what we do. Let's focus our resources on other, more prevalent harms." Unfortunately, such a statement would be suicidal for any politician. ("You mean you're just gonna sit there and watch? Do something, dammit!") The real solution to terrorism is threefold. First, educate people that perfect security is impossible. Second, focus on intelligence and early detection, instead of trying to achieve 100% prevention. Third and most importantly, work to fix the underlying causes why so much of the world hates the US (or the UK, etc).

Security is never "done" -- it is an ongoing process of reevaluation. Schneier likes to think of it as a never-ending game (with high stakes) against an intelligent and evolving adversary. But each of us makes security tradeoffs every day, so we should not "leave the security to the experts". We should recognize when security decisions are made based on fear, instead of a rational analysis. And we should be mindful of people's agendas when they tell us that something is "for security purposes". Perfect security is impossible. In Schneier's words, "security is a tax on the honest," but "the price of freedom is the possibility of crime."

Go get the book. It's awesome. Also check out Schneier's blog and monthly newsletter.

Free Culture

2013-01-15T00:00:00-08:00

Have you ever seen one of those movies where some kid knows a disaster is coming, but no one believes him until it's too late? I got the same helpless feeling from reading Lawrence Lessig's book Free Culture, except that this time the stakes are real. Intuitively, intellectual property is just like physical property, and should be given the same levels of protection. It's easy to accept this proposition without examining it any further, but Lessig argues that it is in fact wrong. Thanks to extreme measures taken to protect intellectual property, we are in danger of becoming a society where only the rich and powerful are allowed to create.

We are already dangerously close. Let's say you're making a documentary film, and want to include a short clip from some other film. (This does not hurt the original creators in any way -- they continue to make money from selling their film.) Your first option is to ask the original creators for permission. They will charge you thousands of dollars for this, and you will have to check with a lawyer to make sure you're not inviting a lawsuit. Your second option is to use the clip without permission, under the fair-use clause. This makes it likely that you will get sued, and even if you manage to defend your case, you're going to spend tens of thousands of dollars on lawyers. Either way, including that short clip in your documentary involves lawyers and big money, so it's out of reach for most independent filmmakers. The sad truth is that today, some forms of creativity come with a hefty price tag.

Lessig contrasts the permissions-and-lawyers culture of the preceding paragraph with the free culture that has been traditional in Western society. When he says "free culture", he does not mean a culture where there is no property, or where creators don't get paid. He means simply a culture where you can borrow ideas freely and build upon them.

Patents and copyrights are the two main mechanisms for protecting intellectual property. They aim to strike a balance between rewarding the original creator and letting society at large benefit from his or her creation. For example, a writer might own the copyright on her book for 14 years, during which she controls the price and gets the profits. (I'm simplifying here.) After 14 years, the book passes into the public domain, and everyone is free to make copies and reuse her work. So the writer gets a monopoly for 14 years, after which everyone is free to use her work for any purpose. At least, this is how things worked in England in the late 1700s.

Today things are more complicated. The copyright term in the US is 90 years. Copyright covers not only copies of books, but all media, including derivative works. Digital technology means that any use of a work, such as viewing it on a website, involves making a copy. For example, if I photocopied a page from a book and put it on the wall in my room, nobody would complain. But if I scanned the same page and put it on my blog, I might be accused of copyright infringement. (Both of these are non-commercial uses -- I am not making any money from that page.) Copyright law today reaches much farther than it did in the past. Lessig summarizes this with a nice chart:

1790	publish	transform
commercial	©	free
non-commercial	free	free

now	copy	transform
commercial	©	©
non-commercial	©	©

On top of all this, a handful of large media conglomerates control about 85% of all newspapers, TV, and radio in the US. This means two things: First, that if they sue you, they can spend a lot more money on lawyers than you can. Second, that they control what a large chunk of the population sees and hears, and if you have a dissenting or controversial opinion, good luck finding someone who is willing to broadcast it.

When a new technology appears, it often enables some kind of "free riding". For example, airplanes fly over your house, but they don't have to ask for your permission. You can record TV shows on your VCR, and you don't have to ask for permission. You can take pictures in public, and you don't have to ask anyone for permission. These are all examples where the law has sought to balance the gain of the new technology with the losses caused by such "free riding" that the new technology enabled. The Internet and peer-to-peer networks are one such technology. But thanks to heavy lobbying by the deep-pocketed content industry, the law is less willing to seek balance this time. Instead, the law seems willing to stifle the new technology in order to protect the business models of the content industry. (In latter years, Lessig switched his focus from copyright law to the influence of money in politics.)

One consequence of the current over-reaching copyright law is that everyone who has ever downloaded an MP3 online (40 million American citizens, or about 13% of the population) are considered criminals. This is a result of the prohibition war fought by the content industry, ostensibly to curb the harm from piracy, but really to make sure their profits keep pouring in. ("Piracy", when defined as the use of someone's creative property without permission, is a lot more complicated than the media industry would like you to believe. Only some of it is harmful, and it is not clear that the overall impact is even negative. Lessig goes into this in depth in the first part of the book.) The content industry doesn't care about protecting artists. Their goal is to make sure that non-commercial creativity doesn't compete, so that all the content out there is theirs to control.

Lessig and his allies have tried a few things to restore sanity when it comes to copyright. First was the Eldred case in the Supreme Court, which was unfortunately lost. Then came proposals for an Eldred act, which were blocked by the media industry's lobby. Lessig seemed to conclude that any change has to come from below, with many people understanding and demanding a free culture. He thus helped launch the Creative Commons movement, which aims to provide a lawyer-free middle ground between the extremes of "no rights reserved" and "all rights reserved".

So if you don't want the rich and powerful few to control all of our culture, go read Lessig's book and familiarize yourself with his arguments. It is a rather depressing book to read, but then again, reality is often depressing. Pick a Creative Commons license that works for you, and use it for your own creations. Encourage others to reuse your work to fuel their own creativity. Start conversations about creative property and piracy that go beyond black-and-white thinking. And if you want to vote with your wallet, Creative Commons is an excellent non-profit to support.

PS. The entire book is available for free, under a Creative Commons license.

Razor Wire

2013-01-06T00:00:00-08:00

This fence surrounds the various antennas on top of Sign Hill.

(more pictures from around South San Francisco)

Nature in Massachusetts

2012-12-01T00:00:00-08:00

I am a terrible procrastinator when it comes to organizing photos. I finally got around to sorting some pictures I had taken in the past few years, when I still lived near Boston. Seeing these pictures again made me miss my escapades into Massachusetts' bipolar but colorful nature.

The Middlesex Fells Reservation, which is unbelievably close to Tufts:

(more pictures from the Fells)

The Great Meadows National Wildlife Refuge, a beautiful place reachable by the Minuteman bike path.

(more pictures from the Great Meadows)

A lake in the Lynn Woods, the only time I ever went there:

(more pictures from the Lynn Woods)

Portrait 69 ("I fell in love in a war zone")

2012-11-21T00:00:00-08:00

She had a voice two octaves deeper than her stature would suggest. Hearing her laugh was like getting shots of dopamine straight into my brain. For the remainder of the ride to school, we talked. We were both strangers to this continent, although she had enjoyed a sizeable head start. Neither of us felt at home here, but there was no point in going back. She told me about the strange lucidity that overtook her on a long-haul flight. I blamed it on sleep deprivation and increased oxygen to the frontal cortex. She preferred to attribute it to something more mysterious.

Her eyes were disarming. Like you're nine years old and playing with your kitten, and you realize you can't even clench your fists. I had to blink if I wanted to breathe. I had to look away if I wanted to hear what she was saying. Perhaps my heart was young then, but never since have I gazed into a pair of eyes that exposed and unsettled me so. Thin made-up shadows were on her eyelids that day, and I asked if she always wore them. She said yes, that makeup was like a way to hide, and that I wouldn't really understand unless I wore it myself. At the time, I couldn't imagine what insecurities someone like her would possibly have to hide.

The shuttle bus reached campus, and we went on our separate trajectories. For the rest of the day, I felt like I was walking on clouds. I was distracted in class. I took the stairs two at a time. I stopped and smelled the magnolia flowers. My mind came to the bittersweet realization that, though I would never have her, she was the most beautiful girl I had ever seen.

I caught glimpses of her every now and then in the months that followed. Here she was in a cerulean dress that sucked the color out of everyone around her. There she was taking off a heavy winter coat, revealing the expensive-looking red velvet on the inside. If I was close enough, I said hello. Sometimes she responded, and that was enough to make a good day. Other times she didn't hear me, and that hurt. I resented her for having such power over me, and I hated my own fickle sense of self worth.

We were profoundly incompatible. My tools were pen and pencil, and my canvas was a sheet of paper. Her tools were shawls and bracelets, and the canvas was her own body. I curled up with a book on Friday nights, while she went out dancing. I was still tongue-tied around girls; she could have anyone she wanted. I imagined myself in the shoes of the guys she hung out with. I knew she was wrong for me, but I still melted at the sight of her.

The following semester she didn't return to school. Through a common acquaintance, I found out that she had taken a year off to do research in Afghanistan. Apparently, she loved the broken places of the world. I added this to my list of unsolved mysteries about her. Picking up the campus newspaper each day, I made a habit of scanning for her byline. I read everything I could find. I learned that more troops were needed in Afghanistan, not because the Americans were losing the war, but because they had taken it upon themselves to rebuild the country's infrastructure. I learned that the sky there was brown, and that the dust of the desert permeated everything.

At some point I graduated, and then I was too busy to keep track of her. Once or twice I saw her features imperfectly rearranged on someone else's face, but by then I wasn't sure how much my memory of her had been touched up. I remember going to an exhibit of anonymous postcards. One in particular depicted the silhouette of a soldier against a fiery sunset. Under his feet, in small white letters, the caption said, "I fell in love in a war zone." And for the longest time, I couldn't shake the feeling that I knew who the author was.

Why would a girl with seductive eyes and a perfect smile give up a world that cherished her femininity, and embrace instead a culture that had it all backwards? I guess her mysteries were never mine to solve. I had escaped from a place of poverty and corruption into a country with too much power and not enough guilt. She had escaped from a place of rampant consumerism and mandatory makeup into a country where she had to wear a burqa. Who am I to say that I made the better choice? In the end, we were both just trying to escape from ourselves.

(image source)

Fort Funston

2012-11-18T00:00:00-08:00

From here we watched for the enemy that never came.
(Motorized vehicles prohibited. Dogs required.)

(more pictures)

The Tangled Web

2012-11-10T00:00:00-08:00

Michal Zalewski's The Tangled Web is an excellent introduction to web application security. The book is structured in three parts. The first part explains how the web works. I thought I knew this stuff, but I was surprised at how many seemingly harmless quirks gain a security significance in the hands of an imaginative attacker. For example, paypal.com and paypaI.com look the same, but are they really? (And by the way, did you know there is a Unicode character called "cat face with tears of joy"?)

Part two dives into the various security features that browsers implement. This includes a comprehensive discussion of the same-origin policy, cookies, frames, content types, and browser plug-ins like Flash. The big picture is rather depressing, as so many complex components inevitably interact in unexpected ways, leading to vulnerabilities. When a new security hole is discovered, it often has to be fixed using a messy workaround, because a drastic security-minded redesign would break backwards compatibility with too many websites. On the bright side, each chapter ends with a "cheat sheet" of practical advice, which makes the web-security landscape a little less daunting.

Part three ends the book with a brief look at upcoming browser features and their security implications. Then there is a two-page epilogue which I found deeply intriguing. Here is an excerpt, where the emphasis is my own:

I am haunted by the uncomfortable observation that in real life, modern societies are built on remarkably shaky ground. Every day, each of us depends on the sanity, moral standards, and restraint of thousands of random strangers--from cab drivers, to food vendors, to elevator repair techs. [...] In this sense, our world is little more than an incredibly elaborate honor system that most of us voluntarily agree to participate in. [...] A degree of trust is simply essential to advancing our civilization at a reasonable pace. [...] What if our pursuit of perfection in the world of information security stems from nothing but a fundamental misunderstanding of how human communities can emerge and flourish? The experts of my kind preach a model of networked existence based on complete distrust, but perhaps wrongly so: As the complexity of our online interactions approaches that of real life, the odds of designing perfectly secure software are rapidly diminishing. Meanwhile, the extreme paranoia begins to take a heavy toll on how quickly we can progress.

Michal Zalewski in The Tangled Web

The rest of this post lists some tidbits from the book that I found particularly interesting.

Tim Berners-Lee wanted to create a semantic web, where chunks of usable information would be enclosed in machine-readable tags, such as <cite>, <code>, and <address>. Most developers don't care about this, and they just use <span> and <div> for everything. As pages become increasingly dynamic and as HTML is reduced to the role of a canvas for CSS and JavaScript, the vision of the semantic web may become irrelevant.
Do not try to sanitize an untrusted document in place. Attackers will always find ways around your safety checks. Instead, parse the document into some in-memory representation (such as a document tree), sanitize that representation, and then serialize it back into a clean and valid document. In the same vein, always use whitelists, because if you use a blacklist, someone will figure out a way around it.
Did you ever wonder why GMail prepends while(1); to all its JSON responses? This is done to protect against a clever way of stealing data by including GMail's JSON on some evil site, after redefining the Array or Object constructor. It turns out that even this might be insufficient, since infinite loops can terminate in JavaScript! Remember those dialogs that say "A script on this page has become unresponsive. Do you want to stop the script or let it continue running?" Zalewski recommends )}]'\n as a "reliable parser-busting prefix" instead.
When you install a browser extension, you often have to wait a few seconds before the "Install" button becomes clickable. This is not there just to annoy you -- it's a security feature! Without it, an attacker could get you to install a rogue extension by showing you a harmless button, and then opening the "install extension" dialog a split-second before you click. This is also probably why there's no keyboard shortcut for downloading a PDF in Chrome, after the "potentially harmful file" warning has been triggered.

Lake Anza

2012-11-04T00:00:00-07:00

After the agonizing climb up the Berkeley Hills, you can feel the temperature drop a few degrees as you enter the woods.

(more pictures)

Force Jekyll to Regenerate Your Entire Site

2012-10-31T00:00:00-07:00

Jekyll is a clever little program for generating static HTML sites. It lets you write your blog posts in markdown, and store them in git. How much more awesome can it get?

By default, when you run jekyll --server, Jekyll will watch your current directory and regenerate the site whenever any file changes. Unfortunately, if one of your plugins has a bug, Jekyll will silently swallow the error. Here is how to force it to regenerate your entire site, and complain loudly about any errors:

rm -rf _site
jekyll --no-auto --no-server

The --no-server option means that Jekyll will quit once it's done regenerating, so you can know when it's finished.

Freedom for the Thought that We Hate

2012-10-09T00:00:00-07:00

I expected this to be a boring book of legalese, but it turned out to be both readable and interesting. The first amendment is short and sweet, but not everyone agrees on how to interpret it.

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

The US Constitution

Anthony Lewis traces the history of how this text was interpreted, from its adoption in 1791 until today. I took free speech for granted in any established democracy, but in fact the US enjoys far stronger protections for free speech than most other countries. For example, in the US it is okay to insult the president, or to burn the national flag. You could go to jail for doing this in other countries. Even some forms of hate speech, such as displaying the Nazi swastika, are protected by the first amendment in the US, while being explicitly banned in parts of Europe.

There are also cases where protection for free speech has perhaps gone too far. For example, donations to political campaigns are deemed protected speech, and thus no limits can be imposed on them. This means that wealthy institutions can spend as much money as they want trying to get favorable politicians into office. Another example is that the press can publish private information about anyone, no matter how intimate or cruel. This puts freedom of the press at odds with personal privacy, and it is not obvious how to balance the two. (Interesting point by Justice Breyer: More privacy would actually encourage free speech, because you are more likely to speak freely if you have no fear that your private conversations will be made public.)

At the other extreme are the instances where the freedom of speech has suffered embarrassing losses. Would you believe that people were once prosecuted for publishing "obscene poetry"? In a more disconcerting pattern, free speech and other civil liberties consistently take a beating every time the US is at war. During such times, politicians manipulate fear to reach for more power, and often encounter little opposition from the public. When the war is over, the injustices and excesses of power are slowly undone. Lewis points out two disturbing ways in which the current "war on terror" is different in this respect. First, because the enemy is so vaguely defined, there is no way to tell when the war will be over. (This reminds me of the endless war in George Orwell's 1984, and the motivations behind it.) Second, the press, which is often the first to cry foul about the government's misdoings, has been unusually quiet this time around.

I liked this book because it gave me a lot of context for why things are the way they are in the US. Despite my general cynicism with regards to the US government, this book gave me a lot of respect for the Supreme Court. Being appointed for life rather than elected, the judges are much more likely to consider the long-term consequences of their decisions, instead of catering to the special-interest groups that pay for most politicians' seats. I was impressed by the depth and acuity of the opinions quoted in the book. I was also glad to learn that the Supreme Court has the maturity to change its mind on an issue every once in a while.

The book treats freedom of speech from the point of view of the constitution, and federal and state laws. Though that would be a large project in itself, I wish that the book had also discussed breaches to free speech in the private sector. For example, Apple, Walmart, and even my own university have been willing, at one point or another, to abridge the freedom of speech in the name of public peace. Some suggest that this will result in a Fahrenheit 451-like society, where speech is repeatedly constrained and reduced to the point of becoming meaningless. With free speech gone, free thought is gone. To make sure that this doesn't happen, we need to cultivate our courage to hear things that make us uncomfortable. E. B. Hall summarized Voltaire's beliefs about free speech in these words: "I disapprove of what you say, but I will defend to the death your right to say it."

PS. If you feel strongly about free speech, consider donating to an organization that promotes and defends this right. Two such organizations are the American Civil Liberties Union and the Electronic Frontier Foundation.

More Yosemite

2012-09-21T00:00:00-07:00

Tuolumne Meadows this time. Clouds Rest trail == not worth it. Cathedral Lakes trail == very worth it.

(more pictures)

Django: Enumerate Objects that Will Get Cascade-Deleted

2012-09-17T00:00:00-07:00

When you try to delete an object, the Django admin gives you a list of all the related objects that will be cascade-deleted as a result. What if you want to get a list of these objects programmatically? You could do it this way:

from django.contrib.admin.util import NestedObjects
from django.db import DEFAULT_DB_ALIAS

obj = ...  # the object you are going to delete
collector = NestedObjects(using=DEFAULT_DB_ALIAS)
collector.collect([obj])
print collector.nested()

This gives you a nested list of the objects that will get deleted if you delete obj. Note that NestedObjects is an undocumented API, so there is no guarantee that it will continue working from version to version. I tested only with Django 1.4.1.

Blown to Bits

2012-09-06T00:00:00-07:00

For better or worse, the Internet is here, and the world must learn to deal with it. Why is this difficult? Because the Internet is unlike anything we've seen before, and it is challenging the assumptions in our heads and in our laws. Let's count the ways in which a digital object -- a song in mp3 -- is different from a physical object -- the same song on a vinyl record:

The vinyl record can be copied at some effort and cost, if you have the right machinery. This is out of bounds for most people. On the other hand, the mp3 can be copied at negligible cost on today's computers, and you almost certainly have a computer. We take scarcity for granted in the physical world, but in the digital world, scarcity is an alien concept.
If the song is controversial enough, a country can impose restrictions on whether the vinyl record can enter or exit its borders. On the other hand, imagine that the mp3 file is on a server in the US, and someone from France (where the song is illegal for some reason) tries to download it. This is much harder to control, even with wholesale censorship. Legal jurisdictions are a fact of life in the physical world, but in the digital world, national borders are artificial.
If I own the only copy of the super-secret vinyl record, I can put it in a safe and hold on to the key at all times. A thief would have to break into my house and cut the safe. But if I put the super-secret mp3 on my laptop, and I use that laptop to browse the web, a skilled thief can break in from another continent. In the physical world, objects have built-in security. In the digital world, security is elusive.
If the thief does get my vinyl record, I can hunt him down and get it back before he makes any copies. But if the thief gets my mp3, and it is interesting to enough people, then copies will mushroom all over the world in a matter of minutes. In the physical world, I can prevent the spread of information by destroying the vehicles that carry it. In the digital world, once the information is out there, you can't get it back.

The examples above are a bit contrived, but you should be able to make the right extrapolations. Just replace "song" with "secret document" or "personal data". The digital world is forcing us to reconsider concept such as copyright, free speech, property, and privacy. Blown to Bits discusses these and other issues in context, explaining where we are and how we got here. I want to dwell on a few things that I found particularly intriguing.

The book claims that we don't really know what we want when we say we want privacy. I think Google Street View is a good illustration of this. Before Street View, if I wanted to look at your house, I had to drive down your street. Now I can just look at your house online without having to drive there. Street View did NOT take something private and make it public -- it only took something already public and made it VERY public. The same thing is happening to public records, such as voting lists. Now anyone can play private detective. With so much information available so easily, you can do some pretty unexpected things, such as de-anonymizing an anonymous dataset. Yet having the dataset publicly available is clearly good for research. So we want the data to be public, and we don't want it to be public. Paradox!

Cryptography is another topic that the Blown to Bits touches upon. To the best of our mathematical understanding, cryptography gives us the tools to exchange encrypted messages that not even a powerful government can decode. Understandably, the US government is uncomfortable with that, and so, for a while, they tried to maintain strict control over cryptography. However, those efforts failed for the most part, and today strong crypto is free to use legally in the US. How did we wrestle this important freedom from the hands of the all-powerful government? According to Blown to Bits, it had everything to do with SSL / TLS, which is the technology your browser uses to securely communicate with e-commerce sites, among others. The world's confidence in secure e-commerce was deemed too important to compromise, and thanks to that, we now have free legal crypto. Very surprising.

Cryptography is also used to provide Digital Rights Management (DRM), a technology designed to prevent piracy. DRM has been largely unsuccessful at achieving that goal. Instead, it forced users to deal with annoying restrictions, such as not being able to transfer their legally purchased content from one device to another. I was pleasantly surprised to read in Blown to Bits that there has been a recent deescalation in the debate over DRM. It all started with an open letter from Steve Jobs to the "big four" music companies, explaining why DRM didn't work and how it hurt the consumer. Now both Apple's iTunes Store and Amazon MP3 sell DRM-free music. (They still protect against piracy through watermarking.) I hope that the deescalation of DRM continues to other forms of content, such as e-books.

Every time I read something on Slashdot, I get depressed about the never-ending encroachments on online freedoms. But, as Blown to Bits shows, not everything is hopelessly broken. The DRM story above is one hopeful example. Another epic win came when the Supreme Court struck down a law attempting to ban "indecent communication" on the Internet. Here is a quote from Justice Dalzell's opinion:

[The] Internet may fairly be regarded as a never-ending worldwide conversation. The Government may not [...] interrupt that conversation. As the most participatory form of mass speech yet developed, the Internet deserves the highest protection from governmental intrusion.

Despite these sudden outbreaks of common sense, we still have a very long way to go. You don't have to look hard to find acronyms that will put a scowl on the face of any Internet freedom fighter. The DMCA that refuses to die, the SOPA / PIPA which (luckily) did die, the TPP that is negotiated in secret... The problem is that laws change too slowly compared to the world of technology, and restrictive laws can chill innovation for years. It does not help that there are powerful interests with very deep pockets out there, who will do everything in their power to maintain control over entire industries.

The digital explosion asks us difficult questions. What should be private and what should be public? What does it mean to have an online identity? How much censorship and surveillance are we willing to tolerate? Do we own our devices and the content on them, or do they belong to the manufacturer / carrier / content producer? Our old metaphors -- ones based on the scarcity of the physical world -- do not work anymore. Will technology be used for freedom, democracy, and innovation, or for control and surveillance? The decision is up to us, and we should join the conversation before it is too late.

PS. If you think the issues discussed in this article are important, consider supporting the organizations that fight for our digital freedoms. Among such organizations are Creative Commons and the Electronic Frontier Foundation.

PPS. Blown to Bits is released under a Creative Commons license. This means that you can download it -- legally! -- for free.

Don't Use CSS Files from raw.github.com

2012-08-28T00:00:00-07:00

The other day I was hunting for a bug in bootstrap, and I wanted to get an older version of their js + css up on jsfiddle for a demonstration. Unfortunately, the bootstrap folks only host the latest version. I thought I could just fetch the js + css from raw.github.com, like this:

<link rel="stylesheet" type="text/css" href="https://raw.github.com/twitter/bootstrap/v2.0.4/docs/assets/css/bootstrap.css">
<script type='text/javascript' src="https://raw.github.com/twitter/bootstrap/v2.0.4/docs/assets/js/bootstrap.js"></script>

The js worked fine, but the css didn't work at all. After some frustrated banging around, I figured out why: raw.github.com serves files as text/plain, and Chrome and Firefox refuse to load css stylesheets with that mimetype.

$ curl -s -D - https://raw.github.com/twitter/bootstrap/v2.0.4/docs/assets/css/bootstrap.css -o /dev/null |grep Content-Type
Content-Type: text/plain; charset=utf-8

This shows up in Chrome's JavaScript console:

Resource interpreted as Stylesheet but transferred with MIME type text/plain:
"https://raw.github.com/twitter/bootstrap/v2.0.4/docs/assets/css/bootstrap.css".

But only on the first page load. On subsequent loads, that URL is cached, and the error is not displayed, making this even more painful to debug.

San Francisco Adventures

2012-08-21T00:00:00-07:00

It is windy out on the water. Prepare to shiver.

(more pictures)

Founders at Work

2012-08-15T00:00:00-07:00

Founders at Work is a book of interviews with startup founders. I finished reading it a few days ago, and I want to collect a few thoughts about it here.

What is the deal with startups? They are a path to financial independence, one of the few remaining forms of freedom in the first world. In the words of Paul Graham, "Instead of working at an ordinary rate for 40 years, you work like hell for four." And then, if you're (very) lucky, you can do whatever the hell you want for the rest of your life, and never worry about paying rent again.

I grew up thinking that starting companies was unfathomably hard, and only some special breed of people could pull it off. There was a brief period in middle school when a certain cheesy book got me excited that maybe working for my entire life wasn't the only option out there. But there wasn't much I could do as a middle-school student, so that thought lay dormant for a long time.

I wasn't forced to think about my future too much until my last year in college, when I was faced with two equally unattractive options. I could go work as a software engineer, which meant I would be little more than a code monkey. Or I could go to grad school, where I would be a poorly paid researcher / slave. I eventually chose the latter, partly because of external influence, and partly because I thought I would get to work on more interesting problems that way. (I still think that going to grad school is worth it, even if one does not intend to become a professor. More on that in a future post.)

Luckily, it was also around that time that I came across this article about Y Combinator. (Thanks V, you know who you are.) While it would be a stretch to say that it made startups look easy, the article made it seem plausible that regular people could start a startup (as opposed to child prodigies, or people who append an MBA to their name). Ever since then, I have become increasingly certain that I want to work on a startup in the not-too-distant future. Founders at Work offers a tiny glimpse at what lies ahead.

Starting a startup is hard -- this was probably the most consistent theme brought up by the people who were interviewed. Founders often isolate themselves from friends, family, and relationships during the early years of a startup. They work non-stop fueled by caffeine, four hours of sleep per night, the promise of wealth, and the even more potent fear of failure. Many asid that they wouldn't have started their company if they knew how hard it was going to be.

Many founders identified perseverance and determination as being the key traits that got them through the early years. This is not so surprising in light of the preceding paragraph. More surprisingly, the original business idea doesn't seem to matter very much. Often the business plan will change substantially as the young company learns what works and what doesn't. This has led various people to proclaim that the team or the market is the main factor determining if a startup will succeed, rather than the original idea. If the founders are determined to succeed, they will eventually land on an idea that works, but often after spending many months working on less successful ideas. Interviewees stressed over and over again that communicating with their users was crucial. If you aren't building something that the users want, you are doomed to having no users to speak of.

While some lucky companies survived and grew with little outside investment, most companies took outside funding from Venture Capitalists (VCs). The interviewees' experience with VCs varied wildly -- from decent VCs who mentored them and their business, to hawk VCs who exploited them and tried to seize control of the business. What struck me most about the funding landscape was how mafia-like the Sillicon Valley really is. The big guns are few, and they all know each other. You get deals through connections, not so much by merit. Your network and your reputation matter a lot.

Another curious fact is that hiring good software entineers in the Sillicon Valley is hard. In the words of Joe Kraus, "great people don't look for jobs, great people are sold on jobs." The demand for good engineers exceeds the supply, and competing for those good engineers means that you are competing against the deep pockets of Google and Apple. As the business grows, it will also need some non-engineers sooner or later. These are sales people, "business development" people, and the like. Engineers often don't know how to evaluate and hire good non-engineers.

Most of the startups in the book have had serious problems with scaling at one point or another. As disconcerting as this may sound, it is almost a necessity when your user base grows by orders of magnitude. Design decisions made with 1,000 users in mind will almost always have to be revisited by the time you reach 1,000,000 users. (On a related note, Mark Fletcher says that internet companies are valued more for their user base then for their technology. On pretending to be bigger than you actually are, James Currier had this to say: "Well, of course you sometimes have to exaggerate. [...] You've got to say you are a step ahead of where you actually are to move to the step that you want to be at.")

The one thing I felt was missing from the book was a discussion of the various controversies in which the startups were involved. For example, there was no mention of TiVo's privacy issues, nor any discussion of the cofounders' opinions on the topic, or the compromises they had to make between user privacy and advertising revenue. Same thing about tivoization and the GPL. It is easy to think of a techie cofounder as "good" and of a big corporation as "bad", but since the latter is often the brainchild of the former, resolving this disconnect requires more nuanced thinking.

Scare of the Day

2012-08-09T00:00:00-07:00

Bike racks are a rare luxury in suburbia. So when I go to the grocery store, I lock my bike to one of those shopping cart thingies:

(image source)

I come out of the store, arms laden with bags, and glance in the direction where I left my bike. This guy is standing there, his expression focused, his hands deftly wielding what looks a lot like my cable lock.

HOLY SHIT HE'S CUTTING MY CABLE!

So I run to him as fast as I can, grocery bags still in hands. As I fill my lungs to scream at him, I look down to see how far he got.

The guy is tying a rope to the shopping carts, probably to keep them in place for the night.

He's NOT trying to steal my bike.

Whew.

That was an awkward O, HAI.

My heart kept pounding for minutes after that.

Monterey Bay, CA

2012-08-07T00:00:00-07:00

Where I learned that I can get a sunburn even through the clouds.

(more pictures)

Yosemite Upper Falls

2012-07-24T00:00:00-07:00

It was like climbing Mount Doom, except we had Clif bars.

(more pictures)

Half Moon Bay, CA

2012-07-15T00:00:00-07:00

Water as cold as sin. A perpetual fog obscuring the sun. The Pacific is a harsh mistress.

(more pictures)

The Things They Carried

2012-05-23T00:00:00-07:00

I probably wouldn't pick up a book if I knew it was about war. There's something very tiring about reading war books -- a kind of helplessness from which you can only hide by becoming cynical. And yet, some of the best books I've read were war stories: The Book Thief, Catch 22, Captain Corelli's Mandolin... It was a good thing I didn't know what they were about when I started reading them. And now the same thing happened with The Things They Carried. Here is a quote I want to remember:

A true war story is never moral. It does not instruct, nor encourage virtue, nor suggest models of proper human behavior, nor restrain men from doing the things they have always done. If a story seems moral, do not believe it. If at the end of a war story you feel uplifted, or if you feel that some small bit of rectitude has been salvaged from the larger waste, then you have been made the victim of a very old and terrible lie. There is no rectitude whatsoever.

I liked the way Tim O'Brien played with fact and fiction, first person and third person, story and meta story. This is definitely a book worth reading.

Thoughts about Artificial-Life Simulations

2012-04-30T00:00:00-07:00

In one of my classes this semester, we programmed little agents in an artificial world to do (hopefully) interesting things. My group focused on exploring disease spreading patterns. We picked a few variables: How fast does the disease spread? Are the agents attracted to one another? Can the agents accurately observe whether other agents are sick? And so on. We built a pretty simple simulation of this artificial world using the excellent MASON simulator. Here's an example of what it looks like:

The green circles are food. The other circles are agents, who are sick (red) or healthy (blue). An agent is perceived as sick by other agents only if it has another smaller dot next to it. The bar above each agent shows the agent's energy; and the agent dies if the energy drops to zero. Agents can eat food to get energy, and they burn more energy per step if they are sick.

Overall, the simulation we built was rather unsatisfying. In this post I'd like to dwell on the reasons why.

We ended up having a lot of parameters (constants), especially for our flocking behavior. How much are agents attracted to food? How much are they repelled by sick agents? How much randomness do we add in their motion? We had about 7 parameters just for the agents' motion, not including simulation-level parameters such as disease type and observability. With this many parameters, there was no systematic way to find good settings for all of them. Instead, we ended up tweaking them until the simulation "looked right". This has been like groping around in the dark, with no clear idea of what we were looking for, and with no way to tell which parameter values were better. Often adjusting the parameters gave us some behavior we wanted, but broke many other behaviors that were working before.

This explosion in the number of parameters seems to be unavoidable in any non-trivial simulation. How might we overcome it? When we don't know the value of a parameter, there should be a way to find a good value automatically. This is where machine learning and search (including evolutionary algorithms) come in. But in order to use any optimization technique, we would need to define what "good" means -- what effect the optimizer should strive to produce. Could we take inspiration from nature, and try to maximize some evolutionary criterion, like the number of offspring an agent has? That would require giving our agents the ability to reproduce. And even in nature, it isn't clear what the "fitness function" is. For example it is not clear that sexual reproduction is better than asexual reproduction, if we are trying to maximize fitness. Yet advanced organisms all use sexual, rather than asexual reproduction. Another example (that Dan Dennett likes to give): Going to college actually reduces your "fitness" -- you will have, on average, fewer children than someone who hasn't gone to college. Yet most of us think that going to college is a good idea ;-) To sum things up, nature is complicated, and it is not clear what "fitness function" evolution is optimizing. Going back to our simulation, we could just optimize some criterion of our choice, instead of trying to emulate nature. But then we might miss out on whatever interesting behaviors could emerge if we had chosen a different criterion to optimize.

Another pet peeve of mine is that we had to hard code the agents' desires, such as the desire to look for food. It would have been more satisfying to give the agents a basic desire (survive!), and let them figure out what to do to achieve it. But how? It seems impossible without giving the agents a general ability to learn, which is beyond what machine learning can do today. In nature, we have basic needs like thirst and hunger, and we also have more complicated drives, like ambition. These needs and drives are not computed in a nice "Needs-and-Drives" module in our brain -- they are the result of a complicated interplay of forces all throughout our bodies. It is very hard to tell what should be "hard coded" in the architecture of an agent, and what should be left for the agent to learn on its own.

Just like we hard coded desires, we also hard coded living costs, such as how much energy the agents burned in each time step. It would have been more satisfying if the agents consumed just as much energy as they needed for the actions they were performing. Computation in nature has its costs: A bigger brain needs more energy to run, so you don't get compute power for free. Reflexes bypass the brain, so latency matters. These aspects are hard to capture in a simulation where time moves in discrete steps. I have a vague idea of charging agents based on the computations they perform (number of instructions executed; bytes of memory used), but it is far from something I could sit down and implement.

So what needs to happen for artificial life simulations to become more than simple toys?

Find a way to navigate the huge parameter space automatically.
Figure out a meaningful fitness function.
Make the costs incurred by agents more realistic.
Find a more realistic way to represent time.

I'd love to read more about this and see what other people have come up with.

Spring Break in NYC

2012-03-27T00:00:00-07:00

You can rent boats in Central Park!

(more pictures)

Adventures of a Library Book Scanner

2012-02-20T00:00:00-08:00

Our library has a bunch of really nice book scanners, which let you save your scan to a USB flash drive, or email the scan to yourself. The scanners are operated by a computer with a touch screen. A normal user probably wouldn't even notice that there is a "real" computer in there, because the touch interface is pretty straightforward to use. That is, until it fails with a helpful error message ("This program has encountered a problem and needs to close"), and dies taking with it everything I've scanned so far.

But that's not the end of the story. When the touch-screen UI dies, it exposes a standard Windows XP desktop underneath. And guess what? It's running under the Administrator account. Okay, keep that in mind in case you want to have some fun with it later.

So I reboot the machine, and the touch-screen UI starts again. I try Alt+Tab, Win+R, Win+D, Ctrl+Shift+Esc; nothing works. I guess the software swallows any keyboard shortcuts to prevent such curious exploration. Oh well. I try to scan my papers again, and again the UI crashes. Fiddlesticks.

Another reboot. This time I hit Win+R before the touch-screen UI is fully loaded, and start an instance of cmd. Sure enough, now I can Alt+Tab between the UI and cmd, and I can launch anything I want from cmd. Let's take a look at the C:\ drive. Whoa, 200 MB free out of 50GB? That might be why it was crashing. So I look around and find the temp directory for the scanner UI, and sure enough it's full of temp files, gigabytes of them. They look like raw bitmaps from previous scans. I delete them all; the scanner UI works again, and I am able to finish my work.

Lessons learned:

The machine runs on an unprotected Windows admin account, wide open for whatever abuse you might want to throw at it.
The scanner UI prevents simple attempts to get around it (like Win+R), but the machine is still vulnerable in the small time window after Windows has finished booting, but before the UI auto-starts.
The UI seems to store raw copies of everything it ever scans. Massive privacy hole. (It's possible that it deletes the temp files if the scan is successful, and only leaves them when it fails, but that's still pretty bad.)
In so many years of desktop computing, we are still caught by surprise when the disk gets full. (A few years back, KDE would fail to log me in without any explanation. It turned out my /home/ was full. Even Mars rovers have failed when their disks got full.) There has to be a better way to handle things. Temp directories that are wiped on reboot are a good first step. Watchdogs might work for desktop users, but not on an embedded system. What else could we do?

Things to Remember when Using ulimit

2012-02-09T00:00:00-08:00

On Linux, ulimit allows you to limit the resources that a process can use. Two use cases:

You have a program that sometimes runs out of memory, slowing your computer down to a crawl. You can use ulimit -v to limit the amount of memory that processes in a shell can use. If a process tries to allocate more memory than that, the allocation will fail and the program will usually abort.
You have a program with a deep recursion, which segfaults with the default stack limit of 8M. You can use ulimit -s to increase the allowed stack size.

There are many more limits you can set; type help ulimit in bash to list them. You can find out the current limits by typing ulimit -a.

Two gotchas that I always forget about:

You may try to limit the memory usage of a process by setting the maximum resident set size (ulimit -m). This has no effect on Linux. man setrlimit says it used to work only in ancient versions. You should limit the maximum amount of virtual memory (ulimit -v) instead.
ulimit has hard limits and soft limits. Hard limits can be decreased but not increased. You can shoot yourself in the foot if you set your hard limit too low. I recommend using soft limits only. Set them with, for example, ulimit -Sv, and query them with ulimit -Sa.

Happy hacking!

Making yEd Import Node Labels from GraphML Files

2012-01-31T00:00:00-08:00

yEd is a gem of a graph editor that makes it very easy to create diagrams and flowcharts. I used Inkscape for this purpose in the past, and had to do a lot of manual alignment. yEd does it all auto-magically with a very intuitive interface. The only bad thing about it is that it's not open source, although it does come free of charge.

yEd can import graphs from a variety of formats, one of which is GraphML. The other day I had a graph in Python (NetworkX), and I wanted to lay it out nicely for printing. yEd's layout functions surpass anything I've seen Graphviz do, so I decided to export the graph to GraphML and load it into yEd. This proved more difficult than I anticipated, but only because I didn't know where to look. Hopefully this post will save you some time.

Let's generate a graph, give the nodes some labels, and export the graph to a GraphML file:

import networkx as nx
graph = nx.gnp_random_graph(10, 0.13, directed=True)
for node in graph.nodes():
    graph.node[node]['label'] = "node %d" % (node + 1)
nx.readwrite.write_graphml(graph, "random.graphml")

Now open the random.graphml file in yEd. All you will see is a square. This is because all the nodes are on top of each other and have no label. Don't despair.

First, let's recover those labels. If you open the GraphML file in a text editor, you will see that NetworkX was smart enough to export the node labels, but yEd was not smart enough to realize they were labels. In fact, they became properties of the nodes in yEd, which you can see by right-clicking on a node, clicking Properties, and then selecting the Data tab.

And now we come to the part that it took me a long time to figure out. We need to map the "Label" property imported from the GraphML file onto the internal property that yEd uses for labels. To do this, click Edit, Properties Mapper. Click the little plus sign under Configurations, and then 'New Configuration for Nodes.' Now click the plus sign next to 'Mappings'. If you're lucky, yEd should figure out what you're trying to do, and automatically select the right mapping. Your window should look like this:

Now select 'Fit Node to Label' if you want the nodes to be resized to fit the labels, then click OK. (If you forgot to select 'Fit Node to Label,' you can do it later by going to Tools, Fit Node to Label.) You should see the labels now:

But the nodes are still on top of each other. To fix that, use one of the algorithms under the Layout menu. Final result:

Ta-da! If you save the modified graph in GraphML format and open the file in a text editor, you will see that reverse-engineering the format that yEd uses to store labels would not be easy.

I Will Never Be Care-Free Again

2011-12-30T00:00:00-08:00

I will never be care-free again. What a sad thing to realize.

These past few days, as I was brainstorming for my senior thesis (a year-long endeavor), I found myself wishing that the work would just stop, and let me relax for a week or two knowing that there are no looming deadlines. This is how it used to be in high school and the first few semesters of college: All my work had an absolute deadline -- the end of the semester. When the break came, I was done, and work wouldn't start again until the next semester. I could relax, knowing that I had no responsibilities until classes started again. I could survive several days without checking my email. I could work on crafts stuff without feeling guilty. My life was structured in alternating periods of work and no-work.

I really miss that.

The work I do now is not like that at all. Research just drags on and on. I'm still trying to finish up some stuff from last summer, and next June I'll be presenting some work that was supposed to be finished one year before. Even though I want to be done with it, the work just keeps coming back; it never ends. If I don't do so well on a problem set, I'll just start fresh on a new one next week. But with research, I have to live with the results and the crappy code and the thorny questions for a long time, and there are no semester cutoffs.

The consequence is that I can't really relax during the breaks, because I always have work that is not finished. I check my email daily, in case something important (work-related) comes up. I feel guilty getting up at 12 and doing origami, because I could be working on my thesis. I end up mixing work and play all day, and then feeling dissatisfied at the end of the day because I was neither productive nor relaxed. My break got shorter by a day, but my pile of work did not get any smaller.

The depressing thing is that grad school will be like this too. The work will leak from one year to the next, never completely finished. And a real job would be the same, I'm sure. The periods I miss, with no work and no responsibilities, are not coming back. I'll never be care-free again, and I need to find a way to relax and rest even knowing that the work will drag on forever.

I'll take a quiet life,
A handshake of carbon monoxide,
No alarms and no surprises please.

Awesome: Edit Encrypted Files Transparently in Vim

2011-12-17T00:00:00-08:00

I just discovered this today: With the gnupg.vim plugin, Vim can edit GPG-encrypted files transparently. So if a file has a .gpg extension, Vim will automatically decrypt it upon opening, and re-encrypt it upon saving. Awesome! I no longer need my clunky script that did this by dumping the cleartext to a temporary file...

The Cost of Applying to Grad School

2011-12-15T00:00:00-08:00

This is a rant about how expensive it is to apply for grad school.

Let's do the math:

$160 to take the GRE General test (which tests high-school-level math with no calculus, ridiculous English vocabulary that only literary people would use, and your ability to write a bullshit essay as fast as possible.)

$140 to take the GRE Subject test (The CS test was very broad; it had questions on everything from networking and operating systems to algorithms and programming languages and RSA encryption.)

$23 x N to send GRE score reports to N schools, assuming you don't use the free four that you get when taking either test (They are sent electronically, so why do they cost so much?!)

$3 x N to send official transcripts to N schools (This probably covers the cost of printing and mailing. Thank you Tufts for not being greedy, although you definitely compensate in other ways. Random fact: some schools don't want an official transcript until you're admitted; some others want two copies for some reason.)

$90 x N average application fee for N schools (Nearly all schools have a higher fee for international applicants. And you can only maybe qualify for a fee waiver if you're a US citizen / resident.)

My total for N=9 schools was $1,236. Holy shit. My bank account is weeping now :( This does not include the psychological costs of lost sleep, ignoring your friends for an entire semester, pounding away at your statement of purpose until your wrists hurt, and constant feelings of inadequacy / anxiety / mild panic. All of it to become... this? Hmm.

How to Find Unused BibTeX Entries

2011-11-20T00:00:00-08:00

BibTeX is a reference management system often used together with the LaTeX typesetting system. Today I wanted to find out if I had any unused references in my BibTeX file. There doesn't seem to be an easy way to do this. Luckily, a combination of tools did it. This is a quick brain dump of what I did. (If you find an easier way to do this, let me know.)

My bibtex file is paper.bib. This contains all (used and unused) references. My paper is in paper.tex. When first compiling the paper, latex creates paper.aux. This intermediary file contains entries only for the references that the paper actually cites.

Dump keys for all (used and unused) references, and sort them:
```
bib2bib paper.bib -ob /dev/null -oc /dev/stdout |sort >all
```

Dump keys for used references, and sort them:

aux2bib paper.aux |bib2bib -ob /dev/null -oc /dev/stdout |sort >used

List keys which are in all but not in used:

diff --old-line-format=%L --unchanged-line-format= all used

You can consult the manual pages for bib2bib, aux2bib, and diff to see what the parameters above do. The commands bib2bib and aux2bib can be found in the bibtex2html package.

Where's My Cheese?

2011-11-12T00:00:00-08:00

Imagine putting error messages on a spectrum, according to how easy to understand they are. "404" lies on the cryptic end. "Page not found" lies somewhere in the middle. I think the middle is a good trade-off between designer effort and user satisfaction. But what lies on the other end? This:

We recognize that our website used to present a challenge, and that many people have memorized the path through the maze or bookmarked the information they need. Unfortunately, due to the new organization of our website content, those trails of breadcrumbs and bookmarks will no longer work. We apologize for "moving the cheese" at the end of the maze, but we think you'll have a much easier time finding the information you need.

Our website content has been organized into a number of related categories, listed below. Please Contact us if you need any further information

The above is a real error message I got on this page, and from a government agency no less. Someone is either over-zealous, or getting paid by the keystroke... The worst problem is that you don't immediately realize this is an error page, so you waste time trying to make sense of that paragraph... Yay usability.

By the way, the Internet Archive can be a great resource when you need to dig up a dead link. If the website is "important" enough (I wonder according to which criterion), the Internet Archive will probably have a stored copy. I've found my article here. Another trick for recently-removed content, or for content that's temporarily down, is to search Google's cache (one, two).

Scraping Ajax websites with Crowbar

2011-10-26T00:00:00-07:00

Sometimes you want to save a local copy of an entire website, either because you want to use it offline, or because you only have access to it temporarily. You could open each page in a browser and save it, but that's tedious. There are nice crawlers out there like HTTrack, which will save an entire website for you, and even tweak the links so that they work on the local version. Unfortunately, such crawlers do not handle JavaScript. If your website uses Ajax to load its content, you're out of luck. Your downloaded copy will contain unexecuted calls like "fetchContent()", instead of the actual stuff that you would see in a browser.

Wouldn't it be great if a crawler could execute the JavaScript on a page, and save it after all the content has been filled in? It turns out that some folks at MIT have already thought about this. Crowbar is a headless Firefox-like browser, running on top of Mozilla's XULRunner. You point it to a URL, it loads it, executes the JavaScript, and gives you the resulting page. It even seems to handle cookies. It's like the full-fledged Firefox running without any screen output, which is exactly what you'd want for web scraping!

After I stopped jumping up and down with excitement, I played with Crowbar a little bit. The idea is great, but the tool itself has some disappointing flaws. First, it hangs if you point it to a URL that is not an HTML page (an image, for example). Why would you want to fetch images with Crowbar? Because some evil websites won't deliver them unless you have the appropriate "session id" cookie. Which brings me to the second flaw: Even though Crowbar handles cookies, there is no way to get at them from the outside. I've found cookies.sqlite in ~/.crowbar/profile-name/, but the database is locked and inaccessible while Crowbar is running. (And even if I could open it, it probably wouldn't store session cookies.) The third flaw is more subtle: When faced with pages that have non-ASCII characters, represented with two bytes in UTF-8, Crowbar seems to silently drop the first byte. This gives you corrupted data, leading to hours of keyboard-smashing frustration.

The last SVN commit in Crowbar was in June 2008, so I am not too optimistic about seeing these bugs fixed. I also don't know enough JavaScript and XUL to do it myself. Still, I think that using a headless browser for scraping is a great idea. I first stumbled upon Crowbar via a post by Jabba Laci. I have since found a few related tools that might be more useful:

PhantomJS, a headless WebKit-based browser with a JavaScript API.
Using QtWebKit directly from Python to scrape pages.
Selenium, which automates browsers for testing. I bet it could be used for scraping, too.

I haven't tried any of these three, but I hold high hopes for the one that uses QtWebKit from within Python. If I understand it correctly, you should be able to get full access over the browser, and peek at cookies, HTTP headers, and anything else you might want. Finally, here are some other random resources that might be useful:

Mechanize, a programmable headless web browser for Python. It doesn't handle JavaScript, but it does handle cookies, and it has a nice interface for filling out forms. No JavaScript means this is much lighter than running a full (albeit headless) browser.
Tools to handle broken HTML: LXML, BeautifulSoup, html5lib.
An interesting blog about web scraping. Also a python library by the same guy.
Apparently there is a fair bit of money and controversy around web scraping.

I began this post talking about crawlers, but then focused on scraping a single page with a JavaScript-enabled headless browser. I don't know about any existing crawlers that support JavaScript / Ajax this way. One problem is that you can't tell when the scripts on a page have finished running. (Crowbar just waits a predetermined amount of time before delivering a snapshot of the page's contents.) Anyway, a JavaScript-enabled crawler sounds like an interesting project :) sudo give me free time...

Choices

2011-10-10T00:00:00-07:00

I shouldn't do this, but I pulled you out for a moment to give you a hint.

Voilà, ma petite Amélie, vous n'avez pas des os en verre. Vous pouvez vous cogner à la vie. Si vous laissez passer cette chance, alors avec le temps, c'est votre cœur qui va devenir aussi sec et cassant que mon squelette. Alors, allez y, nom d'un chien!

Welcome to College!

2011-09-28T00:00:00-07:00

Credits: Humor-sans font by ch00f, based on the handwriting on xkcd.

Glimpses

2011-09-18T00:00:00-07:00

Post mangled, after it has been pointed out to me how it can be misinterpreted as more than a silly writing exercise... Yay self-censorship :( I failed to get across the point that these are (based on) things I've been a reluctant witness to, and not things I've invented...

Original post below:

Out-of-body experiences. Infinite loops. First kisses. Running and laughing in the rain. Caresses that make your toes curl. Falling in love during a two-month internship. Denial of service. Stolen kisses in the library. Hearts breaking. Hearts healing. Connection.

These things happen.

The two of them are locked in a close embrace. Her arms are tight around his neck, pulling him close. Her chin rests on his shoulder. His hand is on the small of her back, holding her near, while his other forearm traces slow circles on her shoulder blades. His cheek presses against the side of her head, and he breathes in the scent of her hair. Other people in the airport stare at them, but their eyes are closed. His forehead is serene. A small unconscious smile lights up her face. The world outside moves noisily on, but there is peace and comfort and trust and gratitude in their embrace. Welcome home.

These things happen.

The sentences in this paragraph start with 'the'. The room is small and the shades are drawn, and the semi-darkness is soft and flattering. The bed is narrow and the mattress squeaks, but it doesn't matter. The average heart rate in this room is twice greater than normal. The average time between inhaling and exhaling is half of what it usually is. The number of samples in the above calculations is two. The portraits on the wall try not to look. The guy next door reaches under his pillow for a pair of earplugs.

These things happen.

A long, white corridor. The light flickers. His palms are flat against the wall, one on either side of her head. She presses her hands against his chest, as if trying to push him away. Their eyes are locked, and their pupils are large. Predator and prey. They both know this is a game of waiting. Their mouths are shaped like two mischievous grins, before they lunge toward each other.

These things happen.

N cnex. N frpyhqrq orapu. Gjb crbcyr, pybgurq. Ur fvgf ba gur orapu. Fur fgenqqyrf uvf ync, snpvat uvz. Tbbq guvat ab bar ryfr vf nebhaq. Fur phcf uvf snpr va ure unaqf, gura yrgf ure svatref jnaqre guebhtu uvf unve. Uvf cnyzf genpr gur fgevc bs rkcbfrq fxva orgjrra gur onpx bs ure wrnaf naq ure gbc. Gurl cynl jub-oernxf-gur-xvff-svefg. Ure purrxf ner syhfurq. Ur hfhnyyl ybfrf. Gurve erpbeq vf sbhegrra zvahgrf naq gjragl-guerr frpbaqf.

These things happen.

You should skip this one. Gur nvecynar vf ybhq, naq gur frngoryg fvta vf ba. Ur yvsgf gur nezerfg frcnengvat gurz, naq fur yrnaf ntnvafg uvf fubhyqre. Ure unaq zbirf sebz uvf xarr, fybjyl, gb uvf guvtu. Ur cynagf n xvff ba ure grzcyr. Gurl erzrzore gur oynaxrg, naq pbire gurzfryirf jvgu vg. Ohg gurer vf fgvyy gbb zhpu gb vasre sebz gur funcr naq zbgvba bs gur pybgu, naq sebz gur grafr rkcerffvba ba uvf snpr. Nsgre n juvyr, ur znxrf n fgenatr pubxvat fbhaq, naq gevrf gb znfx vg jvgu n pbhtu. Gur syvtug nggraqnag cnffrf ol naq ybbxf ng gurz, ohg fnlf abguvat. Gurl fjvgpu ebyrf. Fur pybfrf ure rlrf naq ohevrf ure urnq va uvf purfg. Abj naq gura, fur nyzbfg whzcf bhg bs ure frng, nf vs ryrpgebphgrq. Gur guveq cnffratre va gurve ebj gevrf gb sbphf ba gur obbx ur'f ernqvat.

This is something that happens. Just not to me, apparently. Look! Frogs are falling from the sky.

Doroțcaia

2011-09-07T00:00:00-07:00

Fun times in the countryside.

(more pictures)

Teach Yourself Programming in Ten Years

2011-08-30T00:00:00-07:00

Peter Norvig is director of research at Google, and coauthor of the textbook Artificial Intelligence: A Modern Approach. I recently came across an interesting essay by him, called Teach Yourself Programming in Ten Years. He argues that like anything else, programming takes years to learn, and that books claiming to teach it in 24 hours or 7 days are snake oil.

Other interesting things I found on his website are some thoughts on climate change, and a list of infrequently-asked questions about Python.

What Beasts May Come

2011-08-22T00:00:00-07:00

Some pictures taken recently at a resort near Hîrjauca, Moldova.

Sustainability of Human Progress

2011-08-19T00:00:00-07:00

John McCarthy, one of the pioneers of Artificial Intelligence, has an interesting page on the sustainability of human progress. He argues that we oughtn't be afraid of Earth's increasing population, nor of eventually running out of oil. I was surprised to find out that he is a skeptic about global warming (search for "global warming" on this page). He is now the second prominent scientist that I know of (after physicist Freeman Dyson), who thinks that the fear about global warming is unwarranted.

On the other hand, John McCarthy does say the following:

As I have read more about climate change, I have become convinced that I cannot learn enough about the subject to be entitled to an opinion on my own.

Script to make a video slideshow from a bunch of pictures

2011-08-11T00:00:00-07:00

I wanted to create a video slideshow out of a bunch of pictures. The several GUIs I tried were all disappointing: they duplicated the same picture across many frames, resulting in a large file. Instead, I wanted each picture to use a single frame, and the frames to move very slowly (e.g. 4 seconds per frame, or 0.25 FPS).

Mencoder has a feature to do this, and it lets you specify the output format, resolution, and FPS. For example, we can do:

mencoder mf://*.JPG -mf w=640:h=480:fps=0.25 \
         -ovc lavc -lavcopts vcodec=mpeg4 \
         -o output.avi

There are two annoying limitations (my version is SVN-r1.0~rc3+svn20090426-4.4.3):

It does not support images of different resolutions.
It gets confused by EXIF orientation tags, which are used by most cameras nowadays.

This can result in a cryptic error message:

Unsupported PixelFormat 38 0.00fps Trem:   0min   1mb  A-V:0.000 [366:0]
Unsupported PixelFormat 38
VDec: vo config request - 640 x 480 (preferred colorspace: Unknown 0x0000)
The selected video_out device is incompatible with this codec.
Try appending the scale filter to your filter list,
e.g. -vf spp,scale instead of -vf spp.

Luckily, the convert tool from ImageMagick comes to our rescue. To resize an image to the desired resolution, we can do:

convert orig.jpg \
        -auto-orient -resize 640x480 -gravity center \
        -background black -extent 640x480 \
        result.jpg

The -auto-orient bit rotates the image to neutralize the EXIF orientation tag. The -background black -extent 640x480 bit pads the image with black. This is useful if the aspect ratios don't match, for example if the original image is in portrait, but the desired resolution is in landscape.

The final result: a quick-and-dirty script to resize all images and create the video slideshow: mk_slideshow.sh. It is not the most user-friendly (it could take resolution and FPS as command-line parameters, for example), but it gets the job done.

Bonus tip:

To see the EXIF orientation tag of a JPEG image, we can use the identify command from ImageMagick:

identify -verbose image.jpg |grep Orientation

If it is anything other than TopLeft (or 1), mencoder will get confused.

Have fun making lolcat slideshows ;-)

A New Beginning?

2011-08-02T00:00:00-07:00

I watched Amélie again last night. It is threatening to become my favorite movie. Its carefully crafted layers of cuteness and quirkiness and loneliness and courage, sweetened by Yann Tiersen's music, were as uplifting as always. Ah, if only life could be that interesting...

Anyway, these last few weeks I've been thinking that I want to start writing a blog again. I used to have one in high school, but I pretty much abandoned it since I came to Tufts. I could always blame it on the lack of time, but it wasn't just that. Another superficial reason was that I got on facebook. For a while, school papers were enough to drain my writing impulses, and max-420-letter wall posts were enough to satisfy my erratic bouts of self-expression. The deeper reason, though, was that I was embarrassed by the cheesy stuff I'd written when I was "younger", and I wanted to disassociate myself from it all. Even though there are things I wrote earlier that I still like. (Let's see now: Am I more embarrassed by the immature person I was then, or by how much of that person is still in me? Hmm...)

Fun fact: From starting that blog until leaving home: 240 posts in 602 days, approx 0.40 posts per day. From coming to the US until today: 26 posts in 1073 days, approx 0.024 posts per day. A 16-fold rarefaction in inspiration? Damn, they must have some pretty bad food here :) I think the reason I stopped writing publicly went way beyond embarrassment. It was closer to an expectation of perpetual embarrassment, a defeating feeling that whatever I wrote would be obsolete and laughable in a few years, no matter what. I felt that, because there were so many people around me who were smarter and more adjusted and better writers than me, nothing I could ever say or write or do had anything to offer to the world. So I protected myself, instinctively avoiding anything that was even remotely threatening, including writing. I tried to never let anyone see my flaws, because if they did, they wouldn't want to be around me anymore.

All of the above hasn't made me into a particularly happy guy. (Any reasonable person is probably rolling their eyes "duh" at this point.) It has been slowly dawning on me that there are people who will not run away when you reveal some of your aches and insecurities, but who will listen and give you a hug instead. If I believed in a god, I would thank him (her?) for such people. So what does that mean for writing? It means that there is value in what I have to say, even if I think it's so imperfect. And there is no harm in laughing at it five or ten years from now. Hipolito from Amélie said it this way:

Life is but a draft, an endless rehearsal of a show that will never play.

So why not share the draft with others, while there is still time?

Wish

2011-05-05T00:00:00-07:00

(This is a piece of flash fiction that I wrote for my creative-writing class, backdated accordingly.)

'Don't move.'

Steven's hand glided through the air towards her face, and for a moment she was afraid that he wanted to kiss her. But his fingers went past her chin, did not stop at her cheek, and ignored the mischievous strand of hair that had escaped from behind her ear. Instead, they landed somewhere high on the bridge of her nose, near the eyes, and with a gentle rub were gone again. Smiling from the tip of his extended index finger, she saw the crescent of a black eyelash.

'Make a wish,' he said.

Rachel let out her breath slowly, so as not to betray her relief. She thought for a moment, squinted at Steven's wide-eyed and goofy grin, inhaled a silent wish, and blew on his finger to make the eyelash go away. Then they continued walking.

Steven's voice was as excited as always, and Rachel wondered if he could sense any change in her. She reflected back on the lazy Sundays they spent laughing and arguing about books, amazed at how many both of them had read. She remembered how good it had felt when she realized that Steven was also terrified of dancing, awkward with people, and distrustful of alcohol. He was always the perfect gentleman, and even that time when Rachel cried on his shoulder, Steven comforted her like a father, and did not try to take advantage of her. She remembered how bony and angular his body had felt.

Rachel knew that Steven wanted her. She could sense it in the way his voice quivered and stumbled when he called her on the phone. He always tried to be around, this day weaving her a bracelet of flowers, and the next day buying her a CD that she happened to mention in passing. Today he had invited her to this concert, adamantly rebuffing her attempts to pay for her own ticket. His infatuation was spelled in the way that his face lit up when she ruffled his lopsided hairdo. When Rachel did that, she couldn't help thinking of a poodle.

She wished that she could transform her intellectual connection with Steven into at least an ounce of attraction. If only he were more like Ralph. Ralph, with his towering stature, his deep masculine voice, and his 'whatever' attitude to everything. Ralph, whose mere accidental brush of the hand sent shivers between her legs. Ralph, to whom she had given her virginity the previous night, and who called her a bitch but then she climaxed anyway. She hoped that Steven didn't see the heat in her cheeks.

Rachel wondered how it was possible to melt in the arms of a man who would probably do little more than shrug if she left him. She wondered how it was possible to feel so connected to someone, and yet so unaroused. But Rachel knew that Steven would be there for her no matter how many Ralphs she picked up and discarded. Just like a poodle. So she decided she wouldn't feel guilty about it.

Wheels

2011-04-30T00:00:00-07:00

(This is a piece of flash fiction that I wrote for my creative-writing class, backdated accordingly.)

Oak trees swished by on the side of the road, and he took in their clean scent hungrily. Under his palms, the handlebars became two extra limbs, familiar and light. The tires' skin gripped the asphalt with a satisfying firmness. Every push of his feet on the pedals propelled him farther, and the speed was glorious. This road was his escape. With the forest on one side and the lake on the other, he could pretend to leave behind the daily papercuts of the city. The trucks reversing under his window, with beeps that tore him out of sleep like needles every morning. The drowning feeling of friends and coworkers expecting him to succeed, not to fall behind one step in his track record of excellence. And that Hydra from hell, who always wore a tie, and replaced every finished task with two new ones. He fled away from it all.

At a bend in the road, he saw a man and a woman running side by side on the adjacent footpath. They wore identical clothes, and he wondered if they were a couple. Sometimes, even seeing two people together made his insides clench in sour envy. Today, his younger brother announced that he was getting married. He wished him well, trying to hide the pang of anguish that he felt inside. Some day, he would learn how to talk to women. Some day, he would understand them and know how to make them happy. Some day, he would meet someone and fall in love. But bitter wrinkles were starting to appear in the mirror these days, and he has been saying 'some day' since he was sixteen... He realized that he had been staring at the runners. They caught his gaze and waved a friendly hello. Too slow to react, he rode past them without returning the acknowledgment. And now it was too late to wave back without seeming strange. In the register of his mind, he added two tally marks to the number of people who hated him.

Reaching the north end of the lake, he got off his bike, feeling the fatigue spread from his legs to the rest of his body. The physical effort gave him a vague sense of fulfillment. He made his way through the trees to the hidden clearing on the shore, a quiet and cozy place he had stumbled upon as a child. Here at last was something seemingly detached from age and civilization. He sat down on the grass and took in the sight. The sun had just set, but it still shone red on the scattered clouds, which looked like a furrowed brow in the sky. He heard the lone cry of a seagull. The bird rose with six powerful strokes, then spread its wings into an effortless glide. It swooped down, dipping its beak into the water, then rose again. He remembered sitting outside his childhood home by the dock, watching the seagulls, wondering at the joy and freedom of their flight. When had those emotions abandoned him? Now, he only felt tired.

The Chairs

2011-04-29T00:00:00-07:00

(This is a short story I wrote for my creative-writing class, backdated accordingly.)

The chairs stood uncomfortably side by side, and the space between them measured exactly two inches. On one side of them was the wall, which had not been painted in a long time, and on the other side there was a large beige closet. Underneath, the linoleum floor looked worn and neglected. A tall bookcase towered behind the chairs, its glass panes reflecting the yellow light from the ceiling. And in front of them, a sewing machine was hidden in the bowels of a large auburn box.

Larry and Lydia Hilbert shared a bed in this room, which they folded up during the day to make space. Mrs Hilbert was very proud of her garden on the windowsill. A scented geranium extended its flowery arms to comfort two begonias, which were slightly dizzy gazing down from the eighth-story window. Behind them, a mischievous cactus grinned at the cat, who eyed it suspiciously and never ventured a second sniff. The cat was now perched on top of the bookcase, having gotten there by means of a triple jump that made the chairs groan. They were already laden with all the boys' clothes.

Arthur and George were both nine years old. They shared the only other room in the apartment. George had blond hair, a dimpled chin, and a scream that made their grandfather twist in his grave. He liked to take toys apart and put them back together, and he loved to listen to tales enacted on the radio. Arthur had dark hair, alert brown eyes, and a smile that could charm a mummy. He liked to watch his father crunch equations on paper, and his favorite smell was that of cantaloupes. But most of all, the boys liked bickering with one another. Their chairs were a frequent object of dispute, because there was never enough space between them. One day, George's shirt would extend a fractious sleeve over to Arthur's chair. The next day, Arthur's socks would slip under George's chair in retaliation. Order would be restored after a brief quarrel, only to be broken again in a few days.

The Hilbert brothers divided everything with careful precision. The cabinet in their room had two vertical compartments, where the boys kept all of their toys and school things. Each boy guarded his side ferociously from the other. Puck, their neighbors' dog, could be petted by Arthur on odd days, and by George on even days. He smiled with his tongue out and his eyes slightly narrowed whenever the boys were around. Even the trees in the playground outside were divided. George was not allowed to climb Arthur's trees except by express permission, and vice versa. If Arthur wanted to claim a new tree, then George had the right to claim one too, or he could save his choice for later. Meticulous records of tree ownership were kept.

The Hilberts' life flowed through familiar river beds. Mr Hilbert brought home a paycheck every month, fixed dripping faucets, took out the trash, and warmed the couch reading the newspaper. Mrs Hilbert cooked every day of the week, kept the house clean, did the laundry, and called her mother on weekends. The boys did well in school, and they had not yet reached the woes of puberty. Three things were present at every Hilbert family dinner: the frown on Mr Hilbert's face when he couldn't hear the radio because of the boys' chatter, the tricks that Mrs Hilbert employed to make the boys eat spinach, and the cat's tail cadging for scraps under the table.

It was a warm October evening. Dry fallen leaves were being burnt, filling the air outside with a sour smell. The smoke was mixed with a dense fog, engulfing Mrs Hilbert in a bubble of visibility no larger than a few yards across. Her heart beat anxiously as her feet hurried towards home. She checked her watch. Larry's not gonna like this... Two hours ago she should've been home, should've started working on dinner. By now the boys are probably starving, the cat is meowing himself to death, and her husband is likely calling up the police and the morgue. Who knew that stupid application would take so long to write... But Mrs Hilbert had to submit it today, or the budget wouldn't come through for next year, and she would be out of a job. She wished at least that she could have called home, if only they hadn't removed the telephone to 'cut back on costs and increase employee productivity'...

Mrs Hilbert thought of all the little wrongs that she and her husband did to each other. Small transgressions, never big ones, and unintentional most of the time. Forgetting to take the trash out, or jumping to answer a question that was meant for the other. Silently ignoring a chore until the other one was forced to do it. They were like drops of water accumulating in a glass, each one by itself altogether negligible, and therefore never talked about. Until the glass was filled to the brim, the surface tension broke, and all the little wrongs came back to flood them both... Mrs Hilbert shook her head to disperse the thought.

It was Friday, and she did not look forward to the weekend. Saturday was baking day, and she would break her back for five hours in the kitchen making cheese and potato and apple pies. The boys would gobble them up before they even cooled, and their thanks would be frugal. To them, it would be a Saturday just like any other. But it is worth it, the delight with which the boys devour them is worth it. Then Sunday was cleaning day. She would bend down to sweep away all of the week's dirt, hearing exasperated sighs whenever she asked someone to move over. As if they were doing her a favor. What would they do without me? And after that, a whole new week would start, and between cooking and working and cleaning and helping the boys with maths, Mrs Hilbert would have no time for herself left at all. She checked her watch again, and quickened her pace. How many drops of sorrow am I adding to the glass today? She hoped it wouldn't spill.

The fog made Arthur feel lightheaded, as he looked through the window and couldn't see the ground. Every sound of the city seemed more distant in the grayness, and for a minute he thought that the entire building was floating away from the earth. This did not prevent him from calling George, though, and nitpicking about his clothes, which had crossed chair borders illegally again. The boys did not notice the smell of burning chicken, or the sound of their mother's key unlocking the apartment door. Nor did they see their father's look of annoyance as she came in and complained about work. They only turned their eyes to the kitchen when they heard raised voices.

'But you never do anything! I'm the one who washes the floor and cooks every day and then does the dishes. You--'

Their father's voice was quieter and they couldn't hear.

'Look, all you do is sit on that stupid couch and pick your nose. You burnt the damn chicken, not me. Don't you blame me for coming home late.'

After an unyielding silence, their mother spoke again. Hers was a voice that tried to justify itself, attack, and ask for forgiveness at the same time, almost crying:

'What if I loitered around the house waiting for everything to get done? Who would make the boys' breakfast? What if I didn't pull out the vacuum until the dust was an inch thick? And how would you like the smell of the house if I didn't clean up after the cat? What if--'

The boys were stunned into silence, their squabble about the chairs instantly forgotten. (On top of the bookcase, the cat was very much awake, his whiskers probably wondering if he should worry about tonight's dinner.) If Mr Hilbert turned to look at the boys in that moment, he would see two petrified faces peering out from behind the sewing machine, like soldiers in a wartime trench, looking out into hostile and unfamiliar territory. If Mrs Hilbert turned to look, the boys' faces would evoke to her the moment of freefall, when gravity was suddenly pulled from under you at the top of a rollercoaster ride. But neither parent looked.

George wasn't really sure why he thought of Puck, the dog, at that moment. He remembered when the men from the phone company came to repair a switch on their floor landing. The dog somehow got cornered on the staircase, and he was too terrified to flee down past the two technicians. He stood on the landing and barked and growled like it was the end of the world. Mr Hilbert opened the door to see what was going on, and poor Puck darted straight into their apartment. He had never been inside before, but even that was less terrifying than the noise of the workers' drill.

Mr Hilbert caught hold of their cat and locked him in the bathroom, to avoid a brawl. Puck took refuge under the chair with all of George's clothes. When he put his hand on the dog's nape, George found that Puck was shivering. The boy extended his other hand to pet Puck on the head, but the dog squealed and ducked, as if expecting a blow. Whenever the din of the drill reverberated from the staircase, Puck flattened his ears and released a high-pitched whine. George noticed that the dog's pupils were large, and wetter than usual. He told Puck that it was all right, that he wasn't mad at him for going under his chair.

It was a different memory that entered Arthur's mind when he heard his parents shouting. The Hilberts' apartment had old-fashioned radiators, coated in many layers of white paint, which was now chipping away. Every autumn, when the weather got cold, the municipal heating company did a pressure test before letting the hot water through. Arthur was playing hopscotch in the kitchen, when he looked under the radiator and saw a small pool of black liquid on the linoleum floor. The liquid was viscous and opaque, and Arthur saw his own warped reflection in the darkness. He immediately started crying.

The boy felt that he would be sucked into the black pool, or worse, that the liquid would keep dripping from the radiator, until it covered the entire floor. Then the cat would jump up on a chair and meow in loud alarm. The liquid would keep rising, until it reached his waist, and it would trap him like quicksand. His father would try to open the apartment door, but it would be stuck because of the liquid's pressure. The viscous black ink would reach his shoulders, then his neck. He would have to tilt his head upwards in order to breathe. Then even that would become impossible.

Arthur could feel his heart beating violently, louder than his mother's soothing voice. He kept crying long after she made the dark liquid disappear, and took him in her arms. The fear he felt came from a time before he had words. It was worse than his grandmother's stories, where she was running and hiding and bombs were falling overhead. It was as if Arthur knew that something dreadful was going to happen, and he was trying to tell people, but no one heard or understood what he was saying. It was like those terrible dreams where he searched for his parents, and when he found them, they did not recognize him. Their eyes were too white, and they looked at him like he was a stranger.

The clock on the kitchen wall was shaped like a teapot. Its tick and tock were the loudest sounds at dinner that night. Mr Hilbert kept his eyes down, and he did not turn on the radio. The light from the ceiling danced on his face, and the vertical line between his brows was deeper than usual. He wanted to hear the evening news, but he forsook the radio this time to appease Lydia. He knew that he could turn it back on in a few days.

The four of them were gathered around a small table, and Mr Hilbert desperately wished that they had a larger home. A place with a living room and a dining room, and separate bedrooms for the boys. So they could breathe. So they wouldn't have to step on each other's toes all the time. But no matter how many extra hours he put in, he would never afford it. It was hard enough to buy all their clothes, and secure a modest vacation for the summer. Mr Hilbert wanted to do better, but he couldn't see any way out. Not until the boys grow up and become their own men, at least. He felt that every glance directed at him carried a silent accusation.

Mr Hilbert saw his wife bring her hand to her chest several times, taking deep breaths and closing her eyes, which had red around them. She dropped her fork, and the sound made a strange echo. When her shaking hands reached for the pitcher of juice, he picked it up and refilled her glass. Arthur and George kept quiet, and they ate their spinach without a word of complaint. When they were done, the boys said thank you and retreated to their room. Mr Hilbert tasted irony in their voices, but quickly told himself that he was just imagining it. Arthur did not ask for candy, like he usually did after dinner. Mr Hilbert started doing the dishes, and he saw Lydia opening a book with very thin pages. He also did the dishes the next day. The day after that, the fog dissolved, and the crisp air smelled of the coming winter. Mr Hilbert returned to the couch, and from the top of the bookcase, the cat watched the boys bicker about their chairs.