Replacing newlines in sed

June 22nd, 2006

Replacing newline characters (\n or 0×0a) is something which is not that easy in sed. Apparently, sed prefers to do things one line a time, so manipulating the newline character doesn’t fit into that model particularly well.

A little bird told me how to do it, so here you go, Internet:

How to replace newlines in sed:

sed -n ‘H;${g;s/\n/,/g;p;}’ filename.txt

How to replace newlines in perl:

perl -p -e ’s/\n/,/g’ filename.txt

How to replace newlines using tr:

cat filename.txt | tr ‘\n’ ‘,’

Sorting IP address on the command line

June 15th, 2006

Here are two aliases I always add to my .bash_profile when for working with IP addresses. findip prints all the ips in a file and sortip sorts by IP address in the order you would expect intuitively. If you have no idea what I’m talking about, forget you ever read this.
# try findip -h * | sortip -u
alias findip=’grep -oE “[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}”‘
alias sortip=’sort -t. -k1,1n -k2,2n -k3,3n -k4,4n’

Here’s a guy who has documented a couple dozen cool command line tricks: http://www.pixelbeat.org/cmdline.html

India ends, across the US, into NYC

May 7th, 2006

We got back from India a long time ago, but I haven’t written anything about the last leg of the trip. Refer to Kim’s blog for the details there.

We basically finished out the trip in the following way:

  • Played at Holi in Hyderabad
  • Went to the tea plantations in Munnar
  • Went to a national wildlife sanctuary called Periyar
  • Rented a houseboat for a night
  • Returned to Hyderabad and said goodbye to Shanit’s family

Then we returned to the US, and drove across the country. The trip went like this:

  • Drove from San Luis Obispo, CA to St. George, UT
  • Saw National Parks, including Bryce, Zion, Arches, Canyonlands, Dead Horse Point and the Colorado National Monument (in Colorado)
  • Stayed with Jacob Creed and Amanda Malone in Boulder, CO. Saw Celestial Seasonings Tea factory.
  • Drove through Colorado, Nebraska, Iowa. Stayed at a cool old religious community called the Amana colonies and also visited the Maytag Blue Cheese factory.
  • Stayed with Matt and Jen in Chicago, IL. We originally met these kids through Dan and Ellie – but they were so nice we just had to meet them on the way. :)
  • Drove to Ohio and stayed with Noah and Jenny Levin. Stayed up until 3 AM talking about “Practical Reasoning” with some philosophy Ph.D. students.
  • Drove through Pennsylvania and finally made it to New Jersey.

We then made our way into New York City and stayed with my friend and colleage Aaron Clark in Brooklyn while looking for an apartment to rent in Manhattan. We found a great place and have completely worn ourselves out trying to get settled here. I started work in the new office.

Plastic Packaging

March 12th, 2006

India has a major problem with plastic. Historically, they rarely purchased anything that came in non-biodegradable containers and virtually all trash was food waste or products made from wood, earth, leaves, or other natural materials. Unfortunately, as more and more plastic is being used, the litter is starting to pile up.

In India, there is no stigma against littering. There are billions of pieces of plastic covering virtually every square foot of land in both populated and unpopulated areas. The plastic sometimes blows around in the wind or gets washed into some body of water, but it is basically here to stay.

Even the waste that doesn’t end up as litter is a problem for India. Small piles of garbage are usually set on fire. Larger garbage piles are collected and dumped on the ground (sometimes in a vacant lot in the middle of town, sometimes on the side of the road just outside of town). Between the cows, dogs, other scavengers, and the elements, the natural materials break down quite quickly. Any plastic that hasn’t been burned just lingers forever.

Today roughly half of everything for sale in India is either made of plastic or has a plastic wrapper (my estimate based on observations). There must be alternative forms of packaging and construction that U.S. based producers could start using on their products that they know are going to be sold in countries with limited waste management capabilities. When I get back I’m going to look into this further.

India again: Darjeeling and Kalimpong

March 12th, 2006

We’ve once again left Delhi on the next major leg of our trip. We’re currently in the north east of India, much higher in elevation and in culturally in a different world altogether. This leg of the trip goes like this:

  • Depart Delhi
  • Darjeeling, 2 nights
  • Kalimpong, 2 nights
  • Siliguri, 1 night
  • Return to Hyderabad

Darjeeling is at about 2,100 meters above sea level (or 7,400 ft, depending on which signs you want to believe) in very hilly and mountainous terrain. We’re close to the border of Nepal, and from the sounds of things Bhutan is around here somewhere as well. The Himalayas are all around us, but primarily to the north.

It is so nice to be visiting this part of northern India at this juncture. No offense to anyone who lives in the south, but after getting wrung through the tourist-hounding madness that is Rajasthan, it is incredibly relaxing to be in a region where most people are genuinely friendly, the shopkeepers only rip you off by small fixed amounts, sanitation and education are taken more seriously, and the pace of life is generally a little more sane and slow. Between the cleaner air and the absence of professional tire-pucturers (“puncture walas”) I can feel myself relaxing already.

That is not to say that this leg of the trip has been devoid of any adventure. Quite the contrary – the high mountain roads, typically 7 to 10 ft wide, are always a blast. When the heavy Darjeeling mist rolls in and reduces visibility to 10 ft or so the pedestrians and piles of unused metal bars laying in the road sure do make things interesting. But enough about the infrastructure – really there is nothing here that we can’t deal with easily.

The town of Darjeeling was awesome. The food was great, the tea was great, the hotel was great, and we met a dozen or so American, British, and Australian travelers who were interesting and kind without exception. The town has little bits of British colonial history here and there and Tibetan / Nepalese / Hindu / Buddhist influence in the architecture, culture, and religious sites. The pictures are going to be awesome – thanks in part to the place itself and thanks also to another traveler we met who happens to have the same camera as me and is a huge fan of “Manual Mode” (thanks Gordon!). Highlights from Darjeeling include buying loose-leaf tea (Mike Gratton: you will be receiving some in the mail assuming we can get it back to the US intact), the fellow travelers, and the beautiful mountain setting. We were on “Tiger Hill” at sunrise yesterday, from where we would have been able to see Everest in the distance, but unfortunately it was too cloudy. Clouds are a way of life up here, so there’s no surprise there. Not seeing Everest was a minor let down, but it didn’t really dampen our spirits at all since we’re having such a great time.

We bought 4 seats in a shared jeep (the only way to travel in style) in order to get from Darjeeling to Kalimpong. We saw virtually all of Kalimpong’s major tourist sites today with the help of a very good driver. Again, the pictures will be great. We saw three monasteries, an orphanage complex, two botanical gardens, a paper factory, and some great hilltop views. The downtown section of Kalimpong is dirty and noisy, and every time we walk into a restaurant we’re concerned that we’ll end up getting ill. We’ve stayed pure vegetarian here (as in all of India except Darjeeling) and the food has been absolutely awesome at every meal. “Vegetables Manchurian” and “Chop Suey Noodle Soup” have both been good enough to try to re-create at home. I like Kalimpong a lot.

Tomorrow we’ll relax here and then take another shared jeep down to Siliguri. Reportedly, there isn’t much in Siliguri and we’re only going there so that we’re close to the airport for an 8 AM flight the following morning. We’ll see how that goes though – there are always adventures happening when and where we least expect them.

India Update: Adventures in Rajasthan

March 5th, 2006

Since then, we’ve traveled through some of the major cities in Rajasthan. We flew from Hyderabad to Delhi

After the wedding, we returned to Shanit’s family home in Hyderabad to rest and recover for a day before beginning the next leg of our journey in Rajasthan (which is a state).

This portion of the trip began with a flight to Delhi. Upon arrival, we had a car and driver already waiting for us courtesy of one of Shanit’s uncles (Thanks Saresh!). We kept this driver as we toured Delhi for two or three days (depending on how you count it). The sites here in Delhi are pretty tremendous, including ancient tombs, crumbling and not-crumbling fortresses, and modern Hindu and Sikh temples. If I had to pick one site that stole the show it would be Akshardham (http://www.akshardham.com/). This is a modern temple, remarkable for its craftsmanship more so than its history. It’s amazing what can be accomplished when you have three million man-hours to work with.

The first night we arrived in Delhi we checked in to the Hotel Shelton. Parts of the room were clean, and other parts were not. Rather amusingly, the service at the hotel was pretty dishonest starting from the moment we checked in and they tried to double the nightly rate, to the moment we checked out the clerk tried to charge us a 10% “check-out fee.” It was in a neighborhood called Paharganj, which is basically a giant bazaar. Between having to argue with the hotel staff, attempting to communicate with the not-so-English-speaking driver, and the filth and chaos throughout Paharganj, we only lasted one day. We ended up calling Shubha, one of Shanit’s cousins, and she and her family saved us. The next two nights we stayed with Shubha and her father and mother (Naresh and Usha) while we continued to see the city. For that we are extremely grateful.

We ended up cutting the driver loose because he was expensive, might have been cheating us, and didn’t speak English well enough to satisfy our needs. Unfortunately, the replacement driver we picked up came with his own set of shortcomings, so it’s hard to say whether the switch was worthwhile or not. More on that later.

Leaving Delhi, we drove first to Agra, where we intended to see the Taj Majal (pronounced “Ma-Hell” not “Ma-Hall”) and two other sites (Akbar’s Tomb and something else I can’t remember). Unfortunately, the ~250 km drive took an astonishing 9.5 hours, including a stop to get the spare repaired (why wasn’t it fixed when we were picked up?) and to fix two flat tires. We also hit traffic created by the governor as he was passing through the area. Our final flat occurred roughly 8 km from the Taj Mahal and ended up taking more than 90 minutes to repair since our spare was already shot. So, after a 9.5 hour journey including almost 2 full hours of waiting by the sides of various roads, we thought we were going to the Taj Mahal, but in fact ended up going to a lamp store where the driver assured us his friend (a lamp store employee) was a very good guide and would only charge 100 Rs (though he actually wanted 200). At this point we were both extremely pissed and told this would-be guide to get out of the car and told the driver to take us to the Taj Mahal. After all that, I wouldn’t say the Taj Mahal was really worth it. Yeah, it’s big, and there is a lot of marble, and large parts of it were handcrafted. Unfortunately, neither the visitors nor the air pollution have been kind to the monument, and it was difficult to enjoy considering our day’s ordeal. Perhaps it would have seemed more impressive if we’d not visited Akshardham just the day before.

While we were at the Taj Mahal, something interesting happened. I think Kim may have experienced some sort of fashion or class-based prejudice. Kim was dressed in a cotton lengha, which is basically a fitted shirt, long skirt, and creatively-pinned extra long scarf native to one region in particular (Punjab, I think, but don’t hold me to it). Because of the natural fiber and absence of jeweled handwork, it was more of an “everyday” kind of garment than a piece of formal wear. As we walked around the monument, Kim drew countless icy glares from the women and generally the richer women dished out the iciest glares. Snide remarks, muttered under the breath and in Hindi, were still easy to decipher and took their toll on her (probably even more so because of our already very dour moods). On the other hand the men, particularly the working class men, were so enamored by her that she ended up having dozens of people ask to take their picture with her (and with me also, just to be polite I suspect). I think in total 40 or so pictures were snapped of us that day. We waited for the sun to set on the Taj Mahal because we heard it would be pretty and because the other things we wanted to see had already closed. I’ll post the pictures of this when I get back to the states, but I suspect that there are going to be other pictures of India that are far more breath-taking.

As we left the monument, the driver stopped into a liquor store and picked up a small bottle (375 ml) of booze whiskey. That didn’t really bother us at all, as we will often have a drink after dinner. We made the hair-raising but blessedly short drive from Agra to Fatehpur Sikri at night. Indian roads are the stuff of legend. There are basically cows, goats, bricks, boards, pedestrians, bicycles, and cars going the wrong direction every 50 feet. None of them have reflectors, so going even 20 km is nerve-wracking.

The hotel in Fatehpur Sikri was exceptionally clean. We asked the owner’s son to join us while we ate, and he was very polite and spoke English very well. Unfortunately, the driver had polished off that entire bottle of whiskey and overheard us talking, and since the driver’s English was so bad he had no idea what was being said and decided that the boy was trying to cheat us. When the owner – who was also drunk – heard that they ended up getting into a slow-motion low impact fistfight. At one point I actually had to get between them and break up the fight just to make sure we still had a place to stay and still had a driver the following morning. Unbelievable. I’m not posting the name of the hotel here just because I think the owner’s son is a nice man and I have a feeling he’s the one really running that business. I guess I should warn other traveler’s away from the place, but it is hard to say whether we brought it on ourselves or what.

We awoke bright and early to photograph Fatehpur Sikri at sunrise. This ruined castle complex and mosque was the short-lived seat the Great Akbar. It was pretty and pleasant. We managed to attract the attention of a young Muslim who claimed to be working at the temple but just turned out to be a money-grubbing guide by another name. It was a beautiful place though.

After Fatehpur Sikri went to Jaipur. The historical monuments were not quite as astonishing, but the city itself was very nice. It is a more commercial city, and the shopping opportunities in Jaipur are great. We were also hosted by Amol’s father and mother Agar and Dolly (sp?) Mathur. There was some great food eaten that night, and it was a trip to call Amol from his own house. :) While in Jaipur, we stayed at the Atithi Guest House, which I can recommend to any traveler as safe, clean, and honest.

The drive from Jaipur to Jodhpur is long, something like 350 km. Fortunately, there is a 6 lane divided toll road called National Highway 8 that made the first half of the journey incredibly fast and comfortable. After the first half, we exited the toll road to see a Brahmin temple in a small town called Pushkar just 22 km off the highway. Unfortunately, the driver didn’t go back to the toll road and then on to Jodhpur, but instead took country back roads, converting the ~5 hour drive into a ~9 hour drive. So our driver couldn’t navigate, couldn’t read a map we provided for him, and carried no maps of his own. I told him he should get a map and he said that he didn’t need one because he drives a different place every week!

Jodhpur was not nice. The old city near the clock tower is thick with thieves and scammers. The palace-hotel in town, which is supposed to have a high-end restaurant, wanted to charge us each a Rs 1750 cover charge (which is astronomical considering no meal we’d eaten surpassed Rs 500 total). We went to see the Umaid Gardens that turned out to be little more than a dry grass patch with dozens of men lying around doing nothing. I think the last straw for Kim was when a 12 year old rode by on a bicycle and slapped her breast. I wasn’t feeling well but I was trying to be a good sport, but after that happened we decided to head back to the hotel.

Upon returning to our Jodhpur hotel, I took my temperature and found it to be 99.4 F or so. Over the next few hours it rose to 102.7 F, at which point the hotel manager said he’d call a doctor. Then he forgot to call. So 90 minutes later we walked to hospital that was just across the street anyway. I got some decongestants, antibiotics, anti-acid pills, and something else I can’t remember at the moment. As soon as I took them my fever started dropping off but then I had diarrhea. So I took some Immodium-like medicine and started vomiting. Before the night was out, I had actually #### in the bed for the first and second times in my life. We ran all of the medications by Vivek Chudgar’s mom (Thank you so much!). We decided that staying in Jodhpur was going to be doing us any favors, and that traveling by car wasn’t a good idea since toilets were typically hours apart. With my health improving, we decided to cancel our trip to Jaisalmer and return once again to the comfort of Shubha, Naresh, and Usha’s house in Delhi. Kim booked some plane tickets around 3:00 pm and we were on an Indian Airlines flight at 6:50 pm. We cut our second driver loose at the airport with the understanding that we’d pay for his return trip to Delhi (though amusingly his estimate of the distance is a full 300 km more than any map would indicate – good luck buddy).

In case anyone is wondering how I’m doing, it’s now about 48 hours since my temperate broke 102 F. I have no fever now and no diarrhea or vomiting. I’ve seen two doctors and talked with one on the phone. Buying a ream of medication and visiting the emergency room cost a total of about $13 USD (~ Rs 530). So I’m fine – no need to worry.

I know all this might sound dour but I really am having a wonderful time. What I have not written enough about is about how much Kim and I have enjoyed the hospitality of all of our friends here and of Shanit’s extended family. The people here have been absolutely wonderful (except the con artists).

So now we’re back in Delhi a day or two earlier than originally planned. We’ve been hanging out with Shubha and her cousin Sahil. We’re just going to take it easy, eat, shop, and otherwise relax. On March 8th we’ll begin the next phase of our trip. We’ll be flying to Kerala (where we’ll be getting a houseboat to tour the backwaters) and then on to Darjeeling (tea plantations!).

More later!

Cheers.

Indian Wedding 2

February 28th, 2006

When I last wrote, Kim and I were in attendance at Shanit and Payal’s wedding in Hyderabad.

The multi-day wedding ended in a mind-boggling huge feast with over a thousand guests. The decorations and dress were astonishing, but thankfully the marriage ceremony itself took place late at night and was relatively calm (just close family were in attendance).

I had gathered that in Indian tradition the wife goes to live with the groom’s family following the wedding. I guess I hadn’t really figured out what this meant, because after the vows were said many tears were shed. The Indian men didn’t cry, or if they did, they hid their tears. Perhaps crying is culturally unacceptable for men, or maybe they knew the sad goodbyes were coming and were expecting it. For my part, I was totally blindsided by Payal’s sadness. When she started crying, I cried with her.

For those that know me, I don’t cry often. In the last 15 or so years I have cried on five occasions: once in anger, twice for loss, once in shame, and now once in sympathy. Payal, my heart goes out to you on this occasion and I hope that we can help welcome you as you transition to the next phase in your life.

Update from India

February 22nd, 2006

Kim and I are doing well in India. So far we have spent a day in Singapore (which was exceptionally clean, organized, and humid). While there, we checked out the national orchid museum, which was awesome. We also saw the city a bit, including the “Long Bar” where the Singapore Sling was invented.

Then we came to Hyderabad, which is where Shanit’s family is from (Shanit is a coworker of mine). We spent several days at his family’s apartment and then came out to a resort called Aalankrita, which is basically a pretty, relaxing country resort apart from the city (which is a bit overwhelming). We have take some 420 pictures in the first 5 days or so, so you will all get to see the details. I’m currently writing to you from the business center in the resort, which is the first time we’ve had ready access to email. I don’t expect that we will be making frequent email updates and don’t really have any way of sending the digital prints back via the Internet due to lack of bandwidth…

We heard about the human cases of bird flu in India, but we won’t be going to those cities. Still, we’ll be wary of any birds we encounter. So far, it doesn’t sound like cause for concern. Craig’s mention that there might be some kind of quarantine is worth keeping in mind, though having white skin seems to open every door for us so hopefully it won’t come up or be an obstacle.

The people here have been extremely friendly, and the food has been fantastic (though occasionally a bit spicy). Kim has had some digestive troubles for roughly a day but seems to be improving (Kim says she is very thankful for the pocket tissues, Mom). We’re learning Hindi, starting with yes, no, good, various types of food, beautiful, and are now moving on to vegetables. It’s fun.

Upcoming Plans

January 17th, 2006

This spring holds a world of change for Kim and me. On the to do list:

  1. Move out.
  2. Go to India for 5 weeks, attend a friend’s wedding, and travel around.
  3. Upon returning, drive across the US from San Luis Obispo to New York City.
  4. Find an apartment in New York City and move in.

It can be bewildering at times but is always exciting.

Desert Camping Intro

January 17th, 2006

Some time has passed since my last post. It’s not as though I haven’t been busy though - in fact, just the opposite. I’m not going to write about all of the things I’ve done since my last post, but I would like to mention a few desert camping experiences that someone may find valuable.

Starting somewhere around fall of last year, Kim and I got the desert camping bug. Although our stories might vary in the details, for me it started with a trip to Anza-Borrego. Although it wasn’t our first trip to the Southern California desert parks, it was one of the most enjoyable (at the time). Anza-Borrego is one of the parks that most people in California have never heard of, some have heard of it but don’t know anything about it, and only a few actually drive out to see it. One weekend, feeling frustrated by something going on at the house, we decided to throw the camping stuff into the car and go for a drive not knowing where we were going. We weren’t even heading to Anza-Borrego, but that’s where we ended up. It was amazingly refreshing to get away from our house (lovely though it is) and escape the daily grind only to find ourselves in the crisp dry air of the desert only a few hours later. The smell of sage and tamarisk was curiously restorative, and I could almost feel myself being energized simply breathing the air.

We have touched upon Joshua Tree twice by now but have never camped there. It is admittedly a beautiful park, with interesting flora, fauna, and geology. Unfortunately, there seem to thousands of people from LA that know this as well and hence it is a relatively busy desert park and can be a tough place to camp on the spur of the moment.

On a later weekend we explored Mojave. Mojave is one of those names that most Californians know but practically none of them understand what it means. I wouldn’t have been able to tell you the first thing about the Mojave Desert for the first couple dozen years of my life. Once I finally did realize where it was, it was little more than a barren wasteland of sage and scrub between LA and Las Vegas; something to be passed as quickly and safely as possible. As people drive along the southern edge of this park on the way to Vegas, I’d wager very few of them think about what they are seeing. In my case, I got as far as asking, “Why would anyone live out here?”

A weekend in Mojave changed that perspective forever. This place is 1.7 million acres, with natural caves, lava beds, lava tubes, cinder cones, and good old fashioned sunrises across the mountains. There is a Joshua tree forest here larger and more densely populated than the one in Joshua Tree National Park. We also met some high quality people while here including some very friendly park rangers and naturalist Chris Clarke.

Over Christmas weekend we camped in Death Valley National Park. Again, the size and severity of the park was astonishing. 3.3 million acres of parched desolation, filled with some of the most amazing and uniquely beautiful sights. Everyone will tell you about Badwater, a small salty pool some 282 feet below sea level, home to a unique species of depressed saltwater desert snails. What you don’t hear about as often are places like the Devil’s Golf Course (hundreds of acres of hardened 18” pillars and sheets of dried salt) and a few hidden springs (both in the park and out, such as the beautiful Crystal Spring east of the park).

These recent camping experiences have really had an impact on me. It’s hard to put into words exactly how and why it has changed me, but it has, and I’m glad I got to see it.