Danny O'Brien's Oblomovka

Currently:

2008-08-18»

inbox seventy-two thousand; email2vcard»

Typically my spare time today was spent in the folllowing proportions: 2 hours allotted to emptying my inbox; 3 minutes spent actually emptying my inbox; 1 hour, 57 minutes spent writing a program to theoretically “help” me empty my inbox.

Spreading the procrastination around, here’s that code. It’s for the millions of people who, like me, use good old command line mutt and a graphical contact manager on Unix (although I think it may have more uses than that). It takes a standard email, pulls out the name and email of the person the mail is from, generates a vCard, and then attempts to open that vCard in your nearby contact manager. It uses vobject, hewn from the glowing pits of Chandler, which you can install with easy_install vobject or apt-get install python-vobject. Hours of entertainment.

Things you might be able to glean from this:

my vaguely okay wrapper for command line python utilities, concealed in the Main() class.
vobject is pretty good for handling vCard and iCal objects. I’ve used it a few times.
if you ever want to open a file using the correct handler under GNOME/KDE, xdg-open is your friend.
my ingenious solution for not knowing when the contact handler (in my case, Kontact) will actually pick up the vCard, which is stored in a temporary file that I delete at the end of my program — just wait five seconds, and then delete it. How hacky is that?
that python’s subprocess module badly needs more helper utilities, IMHO. That subprocess.Popen stuff is just Python’s new way of saying “run a shell command”, and it should be clearer than that.

#!/usr/bin/env python
##
# email2vcf.py
###
"""email2vcf.py

"""

__version__ = "0.1"
__author__ = "Danny O'Brien <http://www.spesh.com/danny/>"
__license__ = "GPL v3"

import vobject
import email
from email.Utils import parseaddr
import tempfile, os, subprocess, time


class VcardableMessage(email.Message.Message):
    def toVcard(self):
        r"""
        >>> from  StringIO import StringIO as st
        >>> em = "To: me <me@spesh.com\nSubject: a test\nFrom: Testy McTest <test@spesh.com>\n\nThis is my phone numner: +1 555 134 5678\nThanks!\n\nd."
        >>> p = email.Parser.Parser(VcardableMessage)
        >>> m = p.parse(st(em))
        >>> v = m.toVcard()
        >>> print v.n
        <N{} McTest  Testy >
        >>> print v.fn
        <FN{}Testy McTest>
        >>> print v.email
        <EMAIL{}test@spesh.com>
        >>> print v.note.value
        From email message-id: Unknown
        Dated: Unknown
        Subject: a test
        """
        v = vobject.vCard()
        (name, email)  = parseaddr(self['From'])
        family = None
        givenname = None
        if ',' in name:
            (family, givenname) = name.split(',')
        if ' ' in name:
            (family, givenname) = name.rsplit(' ', 1)
        if not family:
            givenname = name
        v.add('fn').value  = name
        v.add('n').value = vobject.vcard.Name(family = family, given = givenname)
        v.add('email').value = email
        mid = self.get('message-id', 'Unknown')
        dt = self.get('date', 'Unknown')
        sub = self.get('subject','Unknown')
        v.add('note').value = 'From email message-id: %s\nDated: %s\nSubject: %s' % (mid, dt, sub)
        return v

def main(args):
    """ Convert mail to vcard, then kick up systems' vcard handler"""
    if len(args) == 0:
        st = sys.stdin
    if len(args) == 1:
        st = file(args[0],'r')
    p = email.Parser.Parser(VcardableMessage)
    m = p.parse(st)
    v = m.toVcard()
    f = tempfile.NamedTemporaryFile(prefix="email2vcf", suffix=".vcf")
    f.write(v.serialize())
    f.flush()
    p= subprocess.Popen("xdg-open" + " " + f.name, shell=True)
    sts = os.waitpid(p.pid, 0)
    time.sleep(5) # give app long enough to import the data
    f.close()

import sys, getopt
class Main():
    """ Encapsulates option handling. Subclass to add new options,
        add 'handle_x' method for an -x option,
        add 'handle_xlong' method for an --xlong option
        help (-h, --help) should be automatically created from module
        docstring and handler docstrings.
        test (-t, --test) will run all docstring and unittests it finds
        """
    class Usage(Exception):
        def __init__(self, msg):
            self.msg = msg
    def __init__(self):
        handlers  = [i[7:] for i in dir(self) if i.startswith('handle_') ]
        self.shortopts = ''.join([i for i in handlers if len(i) == 1])
        self.longopts = [i for i in handlers if (len(i) > 1)]
    def handler(self,option):
        i = 'handle_%s' % option.lstrip('-')
        if hasattr(self, i):
           return getattr(self, i)
    def default_main(self, args):
        print sys.argv[0]," called with ", args
    def handle_help(self, v):
        """ Shows this message """
        print sys.modules.get(__name__).__doc__
        descriptions = {}
        for i in list(self.shortopts) + self.longopts:
            d=self.handler(i).__doc__
            if d in descriptions:
               descriptions[d].append(i)
            else:
               descriptions[d] = [i]
        for d, o in descriptions.iteritems():
            for i in o:
                if len(i) == 1:
                    print '-%s' % i,
                else:
                    print '--%s' % i,
            print 
            print d
        sys.exit(0)
    handle_h=handle_help

    def handle_test(self, v):
        """ Runs test suite for file """
        import doctest
        import unittest
        suite = unittest.defaultTestLoader.loadTestsFromModule(sys.modules.get(__name__))
        suite.addTest(doctest.DocTestSuite())
        runner = unittest.TextTestRunner()
        runner.run(suite)
        sys.exit(0)
    handle_t=handle_test

    def run(self, main= None, argv=None):
        """ Execute main function, having stripped out options and called the
        responsible handler functions within the class. Main defaults to
        listing the remaining arguments.
        """
        if not callable(main):
            main = self.default_main
        if argv is None:
            argv = sys.argv
        try:
            try:
                opts, args = getopt.getopt(argv[1:], self.shortopts, self.longopts)
            except getopt.error, msg:
                raise self.Usage(msg)
            for o, a in opts:
                (self.handler(o))(a)
            return main(args) 
        except self.Usage, err:
            print >>sys.stderr, err.msg
            self.handle_help(None)
            return 2

if __name__ == "__main__":
    sys.exit(Main().run(main) or 0)

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

#!/usr/bin/env python

# email2vcf.py

###

"""email2vcf.py

"""

__version__ = "0.1"

__author__ = "Danny O'Brien <http://www.spesh.com/danny/>"

__license__ = "GPL v3"

import vobject

import email

from email.Utils import parseaddr

import tempfile, os, subprocess, time

class VcardableMessage(email.Message.Message):

def toVcard(self):

r"""

>>> from StringIO import StringIO as st

>>> em = "To: me <me@spesh.com\nSubject: a test\nFrom: Testy McTest <test@spesh.com>\n\nThis is my phone numner: +1 555 134 5678\nThanks!\n\nd."

>>> p = email.Parser.Parser(VcardableMessage)

>>> m = p.parse(st(em))

>>> v = m.toVcard()

>>> print v.n

<N{} McTest Testy >

>>> print v.fn

<FN{}Testy McTest>

>>> print v.email

<EMAIL{}test@spesh.com>

>>> print v.note.value

From email message-id: Unknown

Dated: Unknown

Subject: a test

"""

v = vobject.vCard()

(name, email) = parseaddr(self['From'])

family = None

givenname = None

if ',' in name:

(family, givenname) = name.split(',')

if ' ' in name:

(family, givenname) = name.rsplit(' ', 1)

if not family:

givenname = name

v.add('fn').value = name

v.add('n').value = vobject.vcard.Name(family = family, given = givenname)

v.add('email').value = email

mid = self.get('message-id', 'Unknown')

dt = self.get('date', 'Unknown')

sub = self.get('subject','Unknown')

v.add('note').value = 'From email message-id: %s\nDated: %s\nSubject: %s' % (mid, dt, sub)

return v

def main(args):

""" Convert mail to vcard, then kick up systems' vcard handler"""

if len(args) == 0:

st = sys.stdin

if len(args) == 1:

st = file(args[0],'r')

p = email.Parser.Parser(VcardableMessage)

m = p.parse(st)

v = m.toVcard()

f = tempfile.NamedTemporaryFile(prefix="email2vcf", suffix=".vcf")

f.write(v.serialize())

f.flush()

p= subprocess.Popen("xdg-open" + " " + f.name, shell=True)

sts = os.waitpid(p.pid, 0)

time.sleep(5) # give app long enough to import the data

f.close()

import sys, getopt

class Main():

""" Encapsulates option handling. Subclass to add new options,

add 'handle_x' method for an -x option,

add 'handle_xlong' method for an --xlong option

help (-h, --help) should be automatically created from module

docstring and handler docstrings.

test (-t, --test) will run all docstring and unittests it finds

"""

class Usage(Exception):

def __init__(self, msg):

self.msg = msg

def __init__(self):

handlers = [i[7:] for i in dir(self) if i.startswith('handle_') ]

self.shortopts = ''.join([i for i in handlers if len(i) == 1])

self.longopts = [i for i in handlers if (len(i) > 1)]

def handler(self,option):

i = 'handle_%s' % option.lstrip('-')

if hasattr(self, i):

return getattr(self, i)

def default_main(self, args):

print sys.argv[0]," called with ", args

def handle_help(self, v):

""" Shows this message """

print sys.modules.get(__name__).__doc__

descriptions = {}

for i in list(self.shortopts) + self.longopts:

d=self.handler(i).__doc__

if d in descriptions:

descriptions[d].append(i)

else:

descriptions[d] = [i]

for d, o in descriptions.iteritems():

for i in o:

if len(i) == 1:

print '-%s' % i,

else:

print '--%s' % i,

print d

sys.exit(0)

handle_h=handle_help

def handle_test(self, v):

""" Runs test suite for file """

import doctest

import unittest

suite = unittest.defaultTestLoader.loadTestsFromModule(sys.modules.get(__name__))

suite.addTest(doctest.DocTestSuite())

runner = unittest.TextTestRunner()

runner.run(suite)

sys.exit(0)

handle_t=handle_test

def run(self, main= None, argv=None):

""" Execute main function, having stripped out options and called the

responsible handler functions within the class. Main defaults to

listing the remaining arguments.

"""

if not callable(main):

main = self.default_main

if argv is None:

argv = sys.argv

try:

opts, args = getopt.getopt(argv[1:], self.shortopts, self.longopts)

except getopt.error, msg:

raise self.Usage(msg)

for o, a in opts:

(self.handler(o))(a)

return main(args)

except self.Usage, err:

print >>sys.stderr, err.msg

self.handle_help(None)

return 2

if __name__ == "__main__":

sys.exit(Main().run(main) or 0)

3 Comments »

2008-08-16»

better living through probability: nationmaster and fivethirtyeight»

Today involved pancakes, the massively co-ordinated garage sale of Bernal Heights, and preventing (and failing to prevent) small child injuries. Now I’m listening to Americans talking about being kids being made to read hefty classics which are thus ruined, like Moby Dick and Ivanhoe and Hemingway and … Ivanhoe? They make them read Ivanhoe here?

Yes, it’s Saturday, and you catch me racking myself over what to write about in this weeks’ column. I think I’d be more peppy if I hadn’t spent far too long last night browsing NationMaster and playing International Man of Goggling At Slightly Dodgy Statistics. Then I stumbled on Visualizing Economics and after spending a few minutes sniggering a bit too much at their plots which look like visualizations of random number generators, got far far too involved in their archive. I started with this depiction of share of world GDP of various countries from 1500-2000 (population over the same period is a nice complement).

For those of you with a matching love of statpron, and about to get overinvolved in US presidential politics (whether through your own choice or that of your dominant media), I heartily recommend Five Thirty Eight which takes a probabilistic approach to the election, running a simulation system 10,000 times and showing the number of times each split in electoral votes occurred in their dice rolls. The tone is slightly Obama-tilted ( with a 5% margin of snark provided by an ex-You Don’t Know Jack editor).

It is a beautifully quantum-multiverse view of the election — as though on the day I will absolutely confident that there will be thousands of universes out there where Alaska went for Obama, or where McCain lost Ohio but won the war: I just won’t know which one I’ll be navigating to. Oh the tiny fluctuations that lead to such endless points of divergence!

Comments Off on better living through probability: nationmaster and fivethirtyeight

2008-08-15»

bandwidth and storage and europe and america»

After doing my reading into fiber in San Francisco, I’d learnt a couple of things: firstly, there’s a lot of fiber around, actually, and secondly, a lot of the fiber under me was owned by Astound, who bought it from RCN, San Francisco’s previously weak-as-tea competition to Comcast.

As it happened, he next day I got some flyers through the letter box from Astound offering 10Mbps for $60 a month. As I’ve been tottering along with 4Mbps/1Mbps for $55 with Comcast, I thought I should look into that.

Competition is a marvellous thing. Wherever in the US Comcast has been facing it, I discovered, they have been magically upping their rates to 16Mbps. Simply calling Comcast support and hinting I was going to shift caused them to mention this fact, and five minutes later, I’m running at around 20Mbps/3Mbps for $65 a month. Add to that the $170 terabyte turned up today, and I feel like I’ve just leapt up about twenty countries in some OECD chart. I guess what I should do now is call Astound and see if they will offer to move PAIX into my bedroom cupboard for $65.99.

Up until now, I’ve always assumed that the UK’s consumer bandwidth situation has been rather better than most of the US — a tidbit gleaned from smug Brit slashdotters, and envy-enducing reports from my friends about their DSL deal-shopping. The received opinion is that the US dropped the ball almost immediately after rolling out broadband, and was promptly outgunned by most of the rest of Europe, something that begging outside telcos for ISDN-level speeds in most of Silicon Valley confirmed for me.

Now, after spending a few minutes on Speedtest’s worldwide self-selected statistics, I’m not so sure. I was originally assuming there was some American-bias to the stats, but digging deeper that doesn’t seem to be the case. The US actually does pretty well compared to Europe (except for those bastards in Sweden, etc) these days.

One thing I’ve learnt is that nation-spanning preconceptions like this are often temporarily true, but not for half as long as they hang around. Pleasantly schadenfreuderish viewpoints have a lot of lag to them. Take mobile phone adoption in the US. When I first arrived here, the difference between US cellphone culture and the UK was stark, and I, like many foreign-media journalists, would frequently dine out on the gap. In 2000, you couldn’t actually consistently text people on other networks; nor would it be reasonable to expect a stranger to have a mobile phone at all.

Then, in I think about 2003, I was crunching some stats about the crappy cellphone penetration in the US for my European friends to gawp at. Instead of doing an us-and-them comparison, I did a time-based one. How far behind was the US chronologically from the UK? It turned out that the US had just crossed 50% of households owning a cellphone. Laughably small compared to 2003 Britain, where it was close to 80% (I am surely misremembering these stats, but bear with me). But nonetheless, pretty much exactly the same as 2000 Britain, my original basis for smugly lording it over the Yanks. America’s primitive phone culture was, it turned out, only as primitive as the futuristic super-advanced one I’d left three years ago.

Sure enough, when my family came to stay that year, my usual prattle about how Americans don’t have mobile phones like we do was swiftly undermined. “What are you talking about, Uncle?” said my annoyingly smart niece, “Look around you. They’ve all got mobiles.”. And they had, and my anecdotes were like drinking from yesterday’s half-empty, cigarette-filled party beers.

I think the same thing is happening with broadband. From 2000-2008 there was a much bigger consumer rollout in Europe than in the US, partly because of government-compelled competition in European telecom (and aren’t I a bad libertarian for even suggesting that), and some really terrible decisions both by business and regulators in the United States. But those differences are slowly closing out as both continents start reaching the limits of DSL and the current infrastructure.

I imagine folks will disagree, which is fine. I’m not entirely sure of the position myself. The real question is: how could we test this? Are the Speedtest stats enough on their own?

6 Comments »

wuala»

Woah, sorry about missing last night: I returned home from work and slept from 8pm to 9am. I blogged in my dreams though.

Briefly, yesterday’s copious free time (ie a few minutes) was spent looking at Wuala (thanks, robwiss!), which is a neat popularisation of some of my pet issues: the infrastructure is a decentralised, fault-tolerant, file storage, with private/public/group access created with a cryptographic filesystem (see the Cryptree paper for details on that, and this Wuala-made Slideshare for a general overview of the tech.) It’s notable for having a user-friendly UI, capability to a run the downloader in a browser via a java client, and therefore have linkability (for instance, in theory you should be able to download the Ogg Vorbis version of the Living in the Edge talk here, once it’s uploaded.) It just went public yesterday, and it’s fun to play around with.

I have a few questions about it, which may be more down to my ignorance than Wuala itself: the source is closed, and so I don’t know yet quite how tied the infrastructure is to Wua.la the company (if Wuala disappeared tomorrow, would the network still exist?), or where the potential weakpoints in overall security might be. On the plus side, Wuala is clearly being used in earnest both for public and private sharing, the user interface does a great job of shielding the crazy cryptopunk shenanigans going on underneath, and it’s cross-platform (albeit via Java, which means it’s not quiite working on my PowerPC Ubuntu server right now).

Tahoe is a lot more transparent, but seems to have a different use case at the moment, which is private nests of stable servers used for distributed backup. But if you wanted to do a free software version of Wuala, that still looks like where you’d start (and Wuala is where you would get your inspiration/learn your lessons from).

4 Comments »

2008-08-13»

the sort of day a horrible stereotype of me would have»

Woke up feeling like the National Sleep Debt, danced with daughter to Fatboy Slim’s remix of magic carpet ride until we were late for school, returned back to struggle (unsuccessfully) with Skype for ORG board meeting call-in, email, run off to record Cranky Geeks, skidaddled to have lunch with Ben Goldacre, who I’d not met before (although am apparently in some big conspiracy with) but I now like a bit too much. Exchanged anecdotes regarding survivalists, on being sued, Americana, assorted data valdez, epidemiology. Had awful suspicion he is as charming as this with everyone. Quick email catch-up, run to pick up daughter. Daughter suggests we explore, get stuck on wrong side of Bernal Heights, shiver as fog closes in, ruefully consider that we may have to survive on only the toy dogs that we find abandoned outside hilltop aromatherapy spas, spend rest of evening trying to find useful link to Fatboy Slim’s remix of “Magic Carpet Ride” to begin this blog with. Wave fist at errant god of consistent metadata and music business models. Talking of which, I am vaguely cheered by continuing leaks of voluntary-collective-license-like systems being considered in UK, although still cautious about technological implementation, privacy issues and other yet unanswered questions, also how close labels always get to this before rearing like frit horses and then galloping off, and, most of all, just how fantastically, painfully, and repetitively smug Andrew Orlowski is going to be if it does come together. Decriminalising millions and saving the music industry will barely make up for it.

Content-free post, so as tradition demands I shall spackle it up with yet another old column from Linux User and Developer, called Danger Dot File. I don’t care what you think, I find these pieces intensely amusing, if only out of amazement that I have lived in an age where one can be paid to write jokes about Unix configuration settings.

Comments Off on the sort of day a horrible stereotype of me would have

2008-08-12»

digging tahoe; wattage update; @fontface!»

After Sunday’s brunch conversation, where we idly dreamt of a capabilities-based distributed-hash-table fault-tolerant filing system, Jeremy took a closer look at Tahoe, Zooko’s project that I’m continually plugging without completely understanding, and noticed that that is exactly what it is.

Last time I played with Tahoe I blundered into a dark alleyway full of unmet python dependencies and got mugged, but the latest version installed a lot more cleanly, and we now have a mini-grid going. I’m also re-reading RFC’s 3401 – 3405 on the Dynamic Delegation Discovery System. I can’t honestly tell you why it feels to me like these are all interlinked, because I’m still at that point where it’s all just pieces looking rather sad on the floor of my garage. I have good instincts, I think, but slow to rationalise, and I backtrack alot. Right now, for instance, my instincts are to put the DiSo people, the AllMyData people, the Zeroconf person, and several bored iPhone hackers in one room, lightly steam them in mild entheogens, and then just when they’re getting somewhere, divert a stampeding herd of straight-edge yet angry and dissolute teenage router-distribution developers into the building. And that can’t be right, so I’ve clearly got some more thinking to do.

I’m going to start dividing up these blog entries when I mention more than one thing, for the love of the metadata. But these are really throwaway items: we discussed how much power a home server draws, and I pulled figures from beyond my arse, but I’ve just run Kill-A-Watt on my old MacMini powerserver (the one running this website), and it does indeed pull down around 17W average. My arse speaks sooth. Of course, I’m not taking into account the cable modem, and the wifi router, so I guess I need to look into that.

In other news, one of the oldest bugs (but not the oldest!) that I watch in Firefox (@fontface support) is getting some new love. Soon, all the major browsers will be able to use downloadable fonts, and then what fresh desktop-publishing linotype hell will we be in?

6 Comments »

2008-08-11»

gmail down; p2p dns»

More fuel for the decentralisation fire with Gmail’s downtime today (Google’s apology). Again, as much as these events people to reconsider keeping all their data marooned on Google’s tiny island in the wider Net, it’s not as if anyone has a more reliable service in place — yet.

It also made me realise that think of another reason why you might want a centralised (or radically decentralised) service that didn’t run on your edge of the Network. Central services are terrible for privacy, but can be better in some contexts for anonymity. Creating a throwaway mail account on a central service (or better still, getting somebody else to), and then using Tor or another anonymising service to access it would provide more temporary anonymity than receiving mail on your own machine (or serving web pages from it). There can also be a big different from serving and hosting data in an authoritarian regime than holding your information remotely in another, more privacy-friendly or remote, jurisdiction. There’s a good reason why a lot of activists use webmail (and why so many were outraged when Yahoo’s webmail service handed over Shi Tao‘s details to the Chinese government).

Tor actually does offer an anonymised service feature, letting you run services from a mystery Tor node, and point to it using a fake domain like http://duskgytldkxiuqc6.onion/. If you were using Tor right now, that would lead you a webpage, served over Tor from an anonymous endpoint. So you can run anonymous services, in theory from the edge. Of course, not everyone is using Tor, so that’s hardly universal provision.

This brings me to another issue that I talked about on Sunday: mapping other non-DNS protocols into the current DNS system. I believe I’ve mentioned before John Gilmore’s semi-serious suggestion a few years back that we grandfather in the current DNS by saying that all current domains are actually in a new, even more top level domain, .icann. — so this would be www.oblomovka.com.icann., allowing us to experiment with new alternatives to DNS, like dannyobrienshomeserver.p2p., or somesuch, in the rest of the namespace.

Other name systems frequently do something like this already: there’s Tor’s .onion fake domain, and Microsoft’s P2P DNS alternative, which resolves to whateveryourhomemachineiscalled.pnrp.net. What neither of those do, however, is have a gateway mapping for legacy DNS users — a DNS server that would respond to standard DNS queries for those addresses, use the P2P protocol to find the IP, then return it to anyone querying using the existing DNS system. That might be a more backward-friendly system than John’s idea.

In Microsoft’s case, that would be pretty easy, even though apparently they don’t do it right now. Resolving .onion in normal DNSspace wouldn’t be possible currently, although I suppose could hack something up (maybe over IPv6, like PNRP) if you were willing to carry all the traffic (and had asked ICANN nicely for the .onion TLD).

I’m not the first person to think that this might be something that would make an interesting Firefox plugin in the meantime.

6 Comments »

2008-08-10»

brunch with bacon; affordable terabytes»

“Why is everyone in San Francisco obsessed with bacon?”.

“The city is built on it. Well, interleaved strata of bacon and vegans”.

I went to a geekhaus brunch, and the bacon was delicious (this is the same group who had successfully created bacon vodka, so it was bound to be.) Rose talked a bit about her research into hacker spaces, most notably the European spaces, but also the parallel US developments like l0ft and NYC Resistor. Jeremy and I spoke a great deal about what the low-hanging-fruit was for the edge-work. I’d initially thought that hacking on making WordPress a platform with some of the functionality of the centralising social-networking sites would be useful, but of course there is already lots of work going on in that space that is already producing results – DiSo and BuddyPress being the two main ones I know about. I need to set aside an evening or two to really grok the standards that DiSo is using.

So instead we talked about the slightly higher-hanging fruit, still unplucked since the last time this was at the forefront of people’s minds. Around then I went looking for smarter people than me to talk to, and ended up crammed into the back of a car with Jonathan Moore,Bram Cohen, and Zooko. I’m really pleased that Zooko is taking a lead on distributed file systems with Tahoe, and I really must drag Jonathan away to talk about all of this, as he’s is one of the most absorbent academic-paper-readers I know. Skimming through some of the XMPP and DiSO discussions, it feels like they’re discussing Zooko’s triangle-style problems right now; I expect they know the reference.

Terabyte drives are down to $175 now. Not quite the $70 I thought we’d have reached by now, back in 2002 — but then, I’m not sure if IBM was factoring in a collapse in the dollar when they published those figures (Matt Daley has a more up-to-date, Australian, perspective) . And we still have the rest of the year to catch up!

1 Comment »

2008-08-09»

Which is more stupid, me or the network?»

I was working late at the ISP Virgin Net (which would later become Virgin Media), when James came in, looking a bit sheepish.

“Do you know where we could find a long ethernet cable?” How long? “About as wide, as, well,” and then he named a major London thoroughfare.

It turned out that one of the main interlinks between a UK (competitor) ISP and the rest of the Net was down. Their headquarters was one side of this main road, and for various reasons, most of the Net was on the other. Getting city permission to run cable underneath the road would have taken forever, so instead they had just hitched up a point-to-point laser and carried their traffic over the heads of Londoners. Now the network was severely degraded, due to unseasonable fog.

The solution was straightforward. They were going to string a gigabit ethernet cable across the road until the fog cleared. No-one would notice, and the worse that could go wrong would be a RJ45 might fall on someone’s head. Now their problem was simpler: who did they know in the UK internet community had a really really long ethernet cable?

I cannot yet work out whether being around when the Internet was first being rolled out is a disadvantage in understanding the terrific complexities and expenses of telco rollout, or a refreshing reality-check. I can’t speak for now, but ten years ago, much of the Net was held together by porridge and string in this way.

(Also, in my experience, most of the national press, and all of the major television networks. All I saw of the parliamentary system suggested the same, and everything anyone has ever read by anyone below the rank of sergeant in the military says exactly the same about their infrastructure. Perhaps something has changed in the intervening ten years. Who knows?)

Anyway, I’ve been reading the San Francisco city’s Draft feasibility study on fiber-to-the-home, which is a engaging, clear read on the potential pitfalls and expenses of not only a municipal-supported fiber project, but any city-wide physical network rollout. I love finding out the details of who owns what under the city’s streets (did you know that Muni, the city’s bus company, has a huge network of fiber already laid under all the main electrified routes? Or that there’s an organization that coordinates the rental of space on telephone poles and other street furniture is called the Southern California Joint Pole Committee?)

It’s also amusing to find out Comcast and AT&T’s reaction to the city getting involved in fiber roll-out:

Comcast does not believe that there is a need in San Francisco for additional connectivity
and believes that the market is adequately meeting existing demand. According to Mr.
Giles, the existing Comcast networks in the Bay Area contain fallow fiber capacity that is
currently unused and could be used at a later date if the demand arises.
…
AT&T does not recognize a need for San Francisco to consider either wireless or FTTP
infrastructure. The circumstances that would justify a municipal broadband project simply do not exist in San Francisco. Service gaps are perceived, not real, according to Mr. Mintz, because AT&T gives San Francisco residents and businesses access to: DSL , T1, and other copper based services from AT&T and Fiber based services such as OptiMAN that deliver 100Mbps to 1 Gbps connectivity to businesses that will pay for it.

My interest in it is more about the scale of any of these operations. The city will take many years to provide bandwidth, and the telco and cable providers are clearly not interested in major network upgrades.

But does rolling out bandwidth to those who need it really require that level of collective action? I keep thinking of that other triumph of borrowed cables and small intentions, Demon Internet, the first British dialup Internet provider, who funded a transatlantic Internet link by calculating that 1000 people paying a tenner a month would cover the costs.

The cost of providing high-speed Internet to every home in San Francisco is over $200 million, the study estimates. But what is the cost of one person or business making a high-speed point-to-point wireless connection to a nearby Internet POP, and then sharing it among their neighbourhood? Or even tentatively rolling out fiber from a POP, one street at a time? I suspect many people and businesses, don’t want HDTV channels, don’t want local telephony, and don’t want to wait ten years for a city-wide fiber network rolled out: they just want a fast cable on their end, with the other end of the cable plugged into the same rack as their servers. And if stringing that cable over the city meant sharing the costs with their upstream neighbours, or agreeing to connect downstream users and defray costs that way, well, the more the merrier. At least we won’t have DSL speeds and be slave to an incumbent’s timetable, and monopolistic pricing and terms and conditions.

I don’t think I would even think such a higglety-pigglety demand-driven rollout would be doable, if I hadn’t seen the Internet burst into popular use in just a matter of months in much the same way. But is the network — and demand — still ‘stupid‘ enough to allow that kind of chaotic, ground-up planning? Monopoly telcos won’t back a piecemeal plan like that for business reasons; cities won’t subsidise it, I fear, because it’s beneath their scale of operation, is too unegalitarian for the public, and undermines their own control of the planning of the city. But if it is conceivable and it is cost-effective, neither should be allowed to stand in its way.

5 Comments »

2008-08-08»

the edge at sxsw»

Liz put forward my “Living on the Edge” talk up for the popular vote South-By-Southwest Interactive (you’re allowed to pressgang other people into appearing, apparently). She let me know I was a contender with a few minutes to go before the deadline. Attempting to re-assert control over my life, I improvised some more proposals: an EFFy event, and one Life Hacks confessional.

As a consequence, it now looks like I’m trying to spam the SXSW panel-picker, and I’m not, no no no I’m not.

Anyway, feel free to vote for me in any combination of ways. Here’s the selection:

Living on the Edge (of the Network)–
How and why we must move to the edge of the network, the edge of the Internet cloud, taking our data out of the hands of centralized corporate services. We can colonize routers and IPv6, phones, and other edges and keep our Internet nodes free and edgy (not my description, but I like it!)
How Not To Be Evil (Even By Accident) – Everyone starts with good intentions (well, almost everybody). But when your Net idea goes huge, or the bottom line lurches, we’re all tempted to cut corners on that hip “don’t be evil” pledge. Panelists from the Electronic Frontier Foundation and elsewhere reveal how you can stop your future self from betraying you, your values — and your users.
Life Hacks Babylon — Gurus Give Their Self-Help Nightmares – Ever wonder if those people blogging about ways to get things done, reach inbox zero, fight procrastination and develop ten new ways to tie their shoelaces are just a little *too* obsessed? You don’t know the half of it. Nightmare tales of self-motivation gone awry from the big names in self-help.

You know, I think people hate self-publicity in blogs, but I realise why people do it. It totally lets you fill up space with something you know vaguely about.

Anyway, here’s what I slapdashedly picked from the 1198 other entries(!), picked without comment (except that the first three are from friends, who I’d vouch for as being particularly good. Or particularly my friends, depending on how you see these things).

Firefox hint: if you hit Ctrl (might be a different control key under Mac or Windows) as you drag over a table, you can select individual cells and columns, and you can paste those cells into forms or spreadsheets and preserve the table data.

Rachel Chalmers, The 451 Group	The Storm Cloud: Business, Infrastructure and Generation Y	Rachel is very very smart, and fascinating topic.
Liz Henry, BlogHer	Open Source Disability Gadgets: DIY for PWD	Liz has lots of great things to say on this.
Jason Schultz, UC Berkeley School of Law	User-Interpreted TOSes: Who Defines the Fine Print?	Jason is always great on this kind of things.
Dennis Dayman, Eloqua	Email Deliverability Secrets from Deliverability.com	I’ve been interested in email deliverability since the Goodmail farrago; it’s sort of the SEO of SMTP, but it gives you an interesting insight into the collateral damage of spam.
Derek Neighbors, Gangplank	Collaborative Development Environments
Samhita Mukhopadhyay, Feministing.com	That’s Not My Name: Beating Down Online Misogyny	Women are getting really smart at dealing with nasty attacks, which isn’t surprising as they’re pretty much the first in line to get them.
Jeff Eaton, Lullabot	Drupal With Its Pants Off	Extra star for title. Drupal is what we use at work.
Andrew Feinberg, Washington Internet Daily / CapitolValley.net	Hackers Meet Hacks: Tech-Friendly Public Policies in the Digital Era	Washington Internet Daily are a pretty good source for inside tech policy wonkiness; it’d be interesting to see a view from inside the beltway.
Tony Shawcross, Denver Open Media	Why NonProfit is the Right Choice for your Startup	One of those “how much do I actually know about non-profit work’ fact-checks. I can never tell whether I’m hopelessly clueless or actually quite experienced. Yes, it is all about me.
Brian Zisk, brianzisk.com	Policy Trainwreck: How Copyright Law Failed the Digital Age	Brian’s a mensch, and has seen this whole battle play out from a great vantage point as both an insider and outsider to the music business. If they’d followed his advice ten years ago, they wouldn’t be screwed.
Chris Bucchere, bdg	Not So Simple Any More: RSS’s Bleeding Edge	Sort of curious about potential new uses for RSS, still. I still struggle to think of how to use RSS to deliver private data between individuals.
Aarron Walter, The Web Standards Project (WaSP)	No Web Professional Left Behind: Educating the Next Generation	Want to get a grip on where WaSP thinks things are heading.
Joshua Baer, OtherInbox.com	Don’t Declare Email Bankruptcy! Take Control of Your Inbox.	Contractual Life Hacks obligatory attendance.
Sandy Jen, Meebo, Inc.	Scaling Synchronous Web Apps	Interesting problem!
Rusty Hodge, SomaFM.com internet radio	Rewriting the DMCA: How to Improve Section 114	Always up for policy suggestions.
Jonathan Dahl, Tumblon	Functional Programming Without A (Strictly) Functional Language	Just like seeing something like this at SXSW. Might well be more interested in the audience than the topic.
Megan McCarthy, Freelance Writer	Surviving Scandal: How to Manage Negative Attention in the Internet Age	I have a feeling I will hate what ends up being said here, but that’s no reason not to go.
Mark Taylor, Level 3 Communications	DRM is Dead	Well, duh.
Patrick Moorhead, Avenue A \| Razorfish	The Invisible Web and Ubiquitous Computing	I shall drop by and see if ubiq still has the smell of 2001 about it.
Larry Chiang, What They Dont Teach You At Stanford Business School.com	What They Dont Teach You At Stanford Business School	I like confessionals.
Martin Kliehm, namics (deutschland) gmbh	The HTML 5 Canvas Element	I love love love Canvas. It’s like having a little Apple ][ in your browser.
Eric Steuer, Creative Commons	Did You Get Ripped Off? Understanding Appropriation	Thinking lots about appropriation at the moment.
Evan Carroll, Capstrat	Who Will Check My Email After I Die?	Why not think about this?
Vince Parr, Globe Telecoms, Inc.	Growing Your Web Traffic Through Mobile Phones in Asia	Don’t have enough anecdata about telephony in asia.
Rebecca Moore, Google	Google Goes to the Amazon	Sounds like a fascinating story.
Jack Moffitt, Chesspark	The XMPP Powered Web	I am all about the Jabber.
Lisa Herrod, Scenario Girl	Aging, Cognition & Deafness: The Quirky Corners of Web Accessibility	Inherently fascinating.
Cyrus Massoumi, ZocDoc, Inc.	Solving the Healthcare Problem…Online	Worthy.
David Marks, Loomia	Privacy and Personalization – Oxymoron or the Perfect Match?	Again, angles on privacy always interest me.
Allen Tom, Yahoo!	OpenID: Foundation of the Social Web	Want to see big OpenID death brawls.
Chris Gammon, Ludorum	Google Runs My Office	Opposing POV to the edge talk.
Tim Hwang, ROFLCon	The State of the Internet Memescape: 2008-10	Funny, I hoop.
Edwin Outwater, EO3 Consulting	Love Thy Archivist for He Will Become a Profit Center	Archivists are always the lynchpin. Hopefully they can escape being turned into ‘profit centers’.
Keely Kolmes, Keely Kolmes Private Psychotherapy Practice	Therapy 2.0: Mental Health for Geeks	Undercovered. Have overpowering personal interest.
Gabriela Schneider, Sunlight Foundation	How the Internet is Transforming Governance	Sunlight are doing really great things right now.
David Crow, Microsoft	The World Isn’t Flat – Building the Next Silicon Valley	I end up having to write a “How do we make X into the next Silicon Valley’ piece every 18 months.
Doc Searls, Berkman Center for Internet and Society	VRM – the Consumer’s Revenge	Doc always has something pertinent to say.
Dan Willis, UX Crank	User Experience 2009: More Crap You Already Know	Title made me laugh.
Jack Moffitt, Chesspark	X Is For XMPP: An Open Messaging Primer	Jabber, jabber, jabber.
Marian Merritt, Symantec	Is Good Cyber Citizenship Really Necessary?	Have a feeling I will agree with everything here, but always like to hear it.
Natasha Sakina Alani, Adaptive Path	Designing Rural Infrastructure: A Euphemism for Money Laundering	Damn true.
Eileen Gittins, Blurb, Inc.	Pursue Your Passion — Leave the Infrastructure to Someone Else	Will try not to be bratty contrarian at back.
Scott McDaniel, SurveyGizmo	How to Become a Google Analytics Expert in a Weekend	Analytics is one of those places where people give more to Google than they get back: I want to see what they get.
Robert Scales, Raincity Studios	Using Drupal to Manage, Publish, and Promote Your Content	Drupal must meet my ignorance head on.
John Athayde, Hyphenated People	Learning from Architecture	Ex-architects are always great conversation-at-3AM people.
Jason Seifer, Rails Envy	OAuth in Every Language	Need… to… understand.. Oauth… better.
Paul Schreiber, Apple	Your Error Messages Suck: Stop Doing That	Will attend for the funny examples.
Glen Campbell, Yahoo! Inc.	Kill the Fail Whale: Scale Your Site	Scaling is fascintating.
Michael Verdi, Millions of Us	Machinima Kung Fu	Have failed to track machinima for last 5 years, want to catch up.
Jeffrey Palermo, Headspring Systems	Managing High-Performing Agile Teams	Vicarious love of agile anecdotage.
John Zeratsky, Google	Getting Things Done the Simple Way	Obligatory Life Hacks attendance.
John Erik Metcalf, Entrepreneur / Conjunctured Coworking / Startup District	Ditch the Valley, Run for the Hills	Love Austin.
Fred Benenson, Creative Commons	Non-Profit Technology Work: How You Can Do Good	Again, want to understand non-profiteryness more.
Heather Champ, Flickr	From Flickr and Beyond – Lessons in Community Management	Heather rules.
John Eckman, Optaros	Open Source and Design: Ideologies Clashing	Current obsession.
Scott Barnes, Microsoft Corp.	Why the F#$k Should I Care About RIA?	Yeah, why the fucking fuck should I?
Derek Gottfrid, The New York Times	Exploiting Massive Parallelism for Fun and Profit	Know the story, want to know more.
Andrew Parker, Union Square Ventures	Hacking Philanthropy	Sort of obligatory, though have yet to hear good version of this story.
Izzy Neis, Six Degree Games	Ask the Moderators: Q&A with Kids Communities Managers	Want to hear realistic take on how “omg think of the children” scenarios are actually handled.
Colin Henry, LiquidPlanner / Apptio	F*cking with Human Backed Web Services	Smells like a hack.
Bhaskar Roy, Qik	Tinkerers Unite: Let Me Show You How it Works	Yay fredom to tinker.
Jakob Heuser, Gaia Online	Unit Testing Back to Front: Prove it Works!	Testing changed my life.
Maria Diaz, Writer	Growing Up as An Internet Oversharer	Public/private dichotomy – you’ll always get me with that.
Alex Hillman, Independents Hall	Do Well by Doing Good – Civic Entrepreneurship	Worthiness.
Margaret Stewart, Google Inc	Herding Cats: The Role of the UX Manager	User experience handling in something I want to know more about.
Jon Wiley, Google	Back Off Man, I’m A Scientist: User Generated Discovery	Title made me laugh.
Jason Reneau, MindBites	Debunking the Free-tard Manifesto: Marketplaces, Content & Rationality	Expect to get really really angry at this. It’s a good emotion!
Joe Solomon, EngageJoe.com	How to Save the World with Firefox Extensions!	Want to know more about FF.
M. Jackson Wilkinson, Viget Labs	Fitting Design and UX into an Agile Process	Again, UX and agile has been a question I’ve had for 5 years.
Thor Muller, Get Satisfaction	Welcome to Your Posthuman Future	Oh, gotta go to the posthumanist SIGs.
Rebecca Fox, mediabistro.com	Why Is Professional Blogging Bloodsport for Women?	See general interest in dealing with trolls.
Dana Loesch, Mamalogues.com / KFTK 97.1 FM	Protecting Your Intellectual Property	They trademarked their name? WTF?
Guy Tennant, Entriq	DRM and Content Providers: A Match Made in Heaven?	Will inevitably be fascinated by this.
Tim Keanini, BayMOO.org	What Can MUDs/MOOs Teach us About Social Technologies?	Will come for the audience.
Danny Kolke, Etelos, Inc.	OpenID, OAuth, Data Portability and the Enterprise	Put off by mention of ‘the enterprise” but Oauth draws me in.
Bruce Henry, LiquidPlanner	Forgetting FTW!	Good choice ‘o’ topic.
Jeremiah Robison, Slide, Inc.	Scale or Die: 10 Lessons About Scaling and Security	Anecdotage on scaling is always good.
Gareth Knight, Kindo.com	Lessons Learned Building Global Apps with Multi-Cultural Teams	International relationships – ongoing obsession.
Tantek Çelik, tantek.com	State of the Microformats	Tantek! The casanova of the div tag!
Tara Hunt, Citizen Agency	Making Whuffie: Raising Social Capital in Online Communities	Tara knows much, good speaker.
Sarah Lefton, UGOBE	Welcome Our Robotic Overlords: The Birth Of New Hot Industry	Robots!
Heather Gold, subvert	Making a Living Being Yourself	Heather!
Amy Hoy, Hyphenated People	How to Be a Glorious Generalist	You had me at everything.
David Crow, Microsoft	How to Demo Like a Demon	I imagine this has stagecraft stuff in it, which I still maintain an interest in.
Fred Benenson, Creative Commons	DRM: The Fight Isn’t Over Yet	Haha, will people please make up their mind?
Jacob Harris, The New York Times	Get Me Rewrite! Developing APIs and the Changing Face of News	Journalism + tech is current obsession.
Anton Kast, Digg	Collaborative Filters: The Evolution of Recommendation Engines	Hmm, if it references academic papers, I’m all in.
Sachin Agarwal, Dawdle.com	Influencing Internet Legislative Changes: Why and How	This sounds deeply in my area.
Lilia Manguy, Avenue A \| Razorfish	Successful Incentive Systems for User-Generated Content	Weirded out enough by this to be interested.
Aral Balkan, Singularity Web Conference	Building on a Cloud	Finding out what people make of cloud infrastructure.
Mark Hines, Ratchet	How Decentralization Impacts Your Content Management Strategy	Decentralization! Gather around everyone!
Clay Johnson, Sunlight Foundation	Coding for Civic Participation	Sunlight out in force.
Eran Feigenbaum, Google	Safety From Above: Cloud Computing and Enterprise Security	More cloud anecdata.
Sean Bonner, seanbonner.com	Change the World in 5 Easy Steps	Have never met Sean, want to.
Doc Searls, Linux Journal, Harvard Berkman Center	Rebuilding the World with Free Everything	Doc!
Jonathan Zittrain, Oxford University; Oxford Internet Institute	The Future of the Internet and How To Stop It	Zittrain!
Dan Hon, Six to Start	We Told Stories – Designing Storytelling Online	Dan!
Joi Podgorny, Ludorum	Setting Up and Running a Remote Team in China	China!
Dan Hon, Six to Start	The BBC, Six to Start and ARGs – Bringing TV to the Web	Dan again!
Traci Fenton, WorldBlu, Inc.	Democracy, Design, and the Future of Work	Democratic workplaces! I haven’t though about that since 1998!
Mason Hale, OneSpot	In the Cloud: Massively Parallel Computing for Everyone	Parrallelism!
Dave Lester, George Mason University	Edupunk: Open Source Education	For the title!
Anthony Berman, Berman Entertainment & Technology Law	Is It Time To Update the DMCA?	So deep in my work thinking.

Peter-Paul Koch, QuirksMode.org	State of the Browsers	Need a recap.
George Kelly, Bay Area News Group-East Bay	Rules for Radicals: Strategic Interventions vs. New Media	Have been reading Alinsky recently.
John Resig, Mozilla Corporation	More Secrets of JavaScript Libraries	Want to brush up on JS…
Jon Wiley, Google	Doctypes Demystified	..and quirks mode weirdness.
Kent Brewster, Yahoo, Inc.	How To Roll Your Own API	Insiders techie view.
WIll Tschumy, Microsoft Corp.	Robots Rock: Human-Robot Interaction	Robots!
Andrew McDiarmid, UC Berkeley	Collectively Licensing File-Sharing at UC Berkeley?	Deep, deep, deep work stuff. Will probably talk in acronyms to Andrew.
Andrew Huff, Gapers Block	The Street is a Platform	Title!
Brad Stenger, Wired	Computational Journalism	This is what I would want to be doing, if I wasn’t having the best time in the world.
Serge Lescouarnec, Serge the Concierge	From Consumed to Thrifty: Strategies for the Good Life in a Wobbly Economy	Plan B

3 Comments »

skip to main bit

Oblomovka

usual, suspect

nearby

info

oblomov

Archives

Currently:

2008-08-18»

2008-08-16»

2008-08-15»

2008-08-13»

2008-08-12»

2008-08-11»

2008-08-10»

2008-08-09»

2008-08-08»