“Democracy, Meh?” →

The New York Times re­cently re­por­ted on a troub­ling stat­ist­ic:

Across nu­mer­ous coun­tries, in­clud­ing Aus­tralia, Bri­tain, the Neth­er­lands, New Zea­l­and, Sweden and the United States, the per­cent­age of people who say it is “es­sen­tial” to live in a demo­cracy has plummeted, and it is es­pe­cially low among young­er gen­er­a­tions.

Fiv­eThirtyEight pos­ted a quick take on the sur­vey that promp­ted that art­icle:

First, the bad news: That data is le­git­im­ate, and it fits with re­search oth­er poll­sters have done on the mil­len­ni­al gen­er­a­tion — loosely defined as every­body born from 1980 to 1997. …

That said, this isn’t ex­actly the same thing as mil­len­ni­als not lik­ing demo­cracy or clam­or­ing for a dic­tat­or, Taylor said[.] … “By any met­ric, gov­ern­ment has per­formed badly” over the last 20 years, he told me. Mil­len­ni­als have come of age in a time when our demo­crat­ic gov­ern­ment has been char­ac­ter­ized by grid­lock, par­tis­an­ship, in­ef­fect­ive­ness and res­ist­ance to change.

20 years of grid­lock is too much. That’s long enough to know that oc­ca­sion­ally vot­ing in dif­fer­ent Re­pub­lic­ans and dif­fer­ent Demo­crats is not go­ing to fix things, as that has happened over those years. We need more. We need to change how we vote, not just who we vote for.

Permalink


“Political Moneyball: America’s Unfair Elections and the Republicans’ Art of Winning Them” →

Jason Kot­tke shares his thoughts on Amer­ica’s elect­or­al sys­tem (and some great C. G. P. Grey elec­tion videos):

More than any­thing for me, this is the story of polit­ics in Amer­ica right now: a shrink­ing and in­creas­ingly ex­trem­ist un­der­dog party has punched above its weight over the past few elec­tion cycles by meth­od­ic­ally ex­ploit­ing the weak­nesses in our cur­rent polit­ic­al sys­tem. Ger­ry­man­der­ing, voter sup­pres­sion, the passing of voter ID laws, and spread­ing pro­pa­ganda via con­ser­vat­ive and so­cial me­dia chan­nels has led to dis­pro­por­tion­ate Re­pub­lic­an rep­res­ent­a­tion in many areas of the coun­try which they then use to ger­ry­mander and pass more re­strict­ive voter ID laws.

Ex­actly right. Take my home state of Utah as an ex­ample. Utah has 4 seats in the fed­er­al House of Rep­res­ent­at­ives. Cur­rently, all 4 of those seats are held by Re­pub­lic­ans. Utah, while (in)fam­ous for be­ing a “deep red” state, is not 100% Re­pub­lic­an. Based on the res­ults of 2016’s statewide elec­tions, Utah is “only” 64%1 Re­pub­lic­an. There­fore, at least one of Utah’s seats should be held by a an­oth­er party, if elec­tions were to be ac­tu­ally rep­res­ent­at­ive of the will of the people.

If you, like me, are con­cerned about where our coun­try is headed after this year’s elec­tion, work­ing to de­feat Trump in 2020 is simply not enough. Par­tic­u­lar can­did­ates like Trump are symp­toms of the prob­lem, not the cause. We need to fix Amer­ica’s demo­cracy at a much more fun­da­ment­al level than just vot­ing out a spe­cif­ic, dis­taste­ful dem­agogue.


  1. To ar­rive at the 64% fig­ure, I av­er­aged the per­cent­age of Re­pub­lic­an votes for all of the 4 Utah statewide of­fices up for elec­tion in 2016: Gov­ernor (66.75%), At­tor­ney Gen­er­al (65.41%), Aud­it­or (63.23%), and Treas­urer (61.23%). As of Monday, Decem­ber 5, 2016, the res­ults have not been cer­ti­fied as fi­nal but I don’t think they will change much. I in­ten­tion­ally did not in­clude the pres­id­en­tial race, as Evan McMul­lin’s in­de­pend­ent run split Utah’s Re­pub­lic­an votes for pres­id­ent. 

Permalink via: Daring Fireball


Donald Trump Fact Checked in the Style of Arrested Development

So good.

via: Kottke.org


Regular Expression Capture Groups in Swift 3 on Linux

I re­cently de­cided to have a go at us­ing Swift. I’ve dabbled with Swift be­fore, run through the odd tu­tori­al but this time I wanted to ac­tu­ally use Swift to ac­com­plish something. I tend to in­tern­al­ize pro­gram­ming lan­guages far bet­ter that way. And since Swift is open source and touted to be a con­tender for a serv­er-side lan­guage, I wanted to try Swift on Linux.

As I got go­ing, I found I needed to use reg­u­lar ex­pres­sion cap­ture groups to find some parts of a string and then re­turn an ar­ray of those parts. In most mod­ern pro­gram­ming lan­guages, this would be a per­fectly straight­for­ward ex­er­cise. Nearly all pro­gram­ming lan­guages these days in­clude a reg­u­lar ex­pres­sion en­gine of some sort in their stand­ard lib­rar­ies and a little Googling will tell you all you need to know about the pe­cu­li­ar­it­ies of that lan­guage’s reg­u­lar ex­pres­sion syn­tax and the APIs for us­ing it. Swift, though, is a bit dif­fer­ent.

Swift’s own stand­ard lib­rary — by which I mean the in­cluded ob­jects and func­tions that are Swift from the ground up and are not de­rived from Ob­ject­ive-C pre­de­cessors — is ac­tu­ally quite small. It con­sists mostly of built-in types like Ar­rays and Strings. The rest of the stand­ard lib­rary, in­clud­ing the reg­u­lar ex­pres­sion en­gine, ac­tu­ally con­sists of bridges to Ob­ject­ive-C ob­jects. As far as I can tell, bridging to Ob­ject­ive-C ob­jects works just fine for the most part; Swift was de­signed, after all, to eas­ily in­ter­op­er­ate with Ob­ject­ive-C. But there are corner cases where frus­tra­tion flour­ishes.

To get star­ted, I found a Stack Over­flow an­swer which pro­posed the fol­low­ing Swift 3/Xcode 8 com­pat­ible func­tion for get­ting the match­ing sub­strings:

func matches(for regex: String, in text: String) -> [String] {
    do {
        let regex = try NSRegularExpression(pattern: regex)
        let nsString = text as NSString
        let results = regex.matches(in: text, range: NSRange(location: 0, length: nsString.length))
        return results.map { nsString.substring(with: $0.range) }
    } catch let error {
        print("invalid regex: \(error.localizedDescription)")
        return []
    }
}

The main idea here is to cast the Swift String in­to an Ob­ject­ive-C NSString. NSString could then give me the NSRange ob­jects that need to be passed to NSRegularExpression. This code does in­deed com­pile and ex­ecute cor­rectly on m⁠a⁠c⁠O⁠S us­ing Xcode 8.

On Linux, however, I im­me­di­ately ran in­to prob­lems. The first is that Swift would ab­so­lutely not al­low me to use the name NSRegularExpression; I had to use the munged name RegularExpression. (The idea seems to be that if there are Swift-nat­ive equi­val­ents — as String is a Swift-nat­ive equi­val­ent to NSString — then you can use the NS pre­fix to still refer to the Ob­ject­ive-C ver­sion. If there is not a Swift-nat­ive equi­val­ent — as is the case with NSRegularExpression — then you must al­ways re­move the NS pre­fix.)

func matches(for regex: String, in text: String) -> [String] {
    do {
        let regex = try RegularExpression(pattern: regex)
        let nsString = text as NSString
        let results = regex.matches(in: text, range: NSRange(location: 0, length: nsString.length))
        return results.map { nsString.substring(with: $0.range) }
    } catch let error {
        print("invalid regex: \(error.localizedDescription)")
        return []
    }
}

Once that was fixed, I got er­rors about miss­ing ar­gu­ments for RegularExpression and its matches meth­od. Adding the options ar­gu­ment to both calls was easy though it re­mains curi­ous that they would be re­quired on Linux but not on ma­cOS.

func matches(for regex: String, in text: String) -> [String] {
    do {
        let regex = try RegularExpression(pattern: regex, options: [])
        let nsString = text as NSString
        let results = regex.matches(in: text, options: [], range: NSRange(location: 0, length: nsString.length))
        return results.map { nsString.substring(with: $0.range) }
    } catch let error {
        print("invalid regex: \(error.localizedDescription)")
        return []
    }
}

The more vex­ing prob­lem was that I got an er­ror re­gard­ing the con­ver­sion of String to NSString: “cannot convert value of type 'String' to type 'NSString' in coercion”. Ini­tially, I as­sumed that NSString was simply not avail­able on Linux and I went down quite a rab­bit hole of try­ing to get an NSRange from String so that I could avoid us­ing NSString al­to­geth­er. It turns out, however, that do­ing so was un­ne­ces­sary. NSString is in­deed avail­able in Swift on Linux and al­though the type co­er­cion mys­ter­i­ously doesn’t work, in­stan­ti­at­ing an NSString manu­ally does:

func matches(for regex: String, in text: String) -> [String] {
    do {
        let regex = try RegularExpression(pattern: regex, options: [])
        let nsString = NSString(string: text)
        let results = regex.matches(in: text, options: [], range: NSRange(location: 0, length: nsString.length))
        return results.map { nsString.substring(with: $0.range) }
    } catch let error {
        print("invalid regex: \(error.localizedDescription)")
        return []
    }
}

I’m not sure why these dif­fer­ences, in par­tic­u­lar, ex­ist between com­pil­a­tion on the two plat­forms. The Linux ver­sion of the code does com­pile and run on ma­cOS with the sole ex­cep­tion of hav­ing to swap NSRegularExpression for RegularExpression.


“Is Donald Trump Actually Trying to Win?” →

John Gruber makes the case that Don­ald Trump isn’t ac­tu­ally try­ing to win the elec­tion for Pres­id­ent:

Nate Sil­ver, “Trump Is Doub­ling Down on a Los­ing Strategy”:

So it’s not sur­pris­ing that Trump has un­der­taken a ma­jor shakeup of his cam­paign, hir­ing Ban­non and pro­mot­ing the poll­ster Kel­ly­anne Con­way. Cam­paign Chair­man Paul Man­a­fort has ef­fect­ively been de­moted. But rather than make a much-ex­pec­ted “pivot” to­ward gen­er­al elec­tion voters — as Man­a­fort had re­portedly been push­ing for — the new plan is to “let Trump be Trump,” doub­ling down on the strategies that Trump used to win the nom­in­a­tion, in­clud­ing an em­phas­is on na­tion­al­ism, pop­u­lism and “bru­tal fights with Clin­ton”.

[…]

In short, Trump isn’t try­ing to ap­peal to more people, which is how you win elec­tions. He’s try­ing to ap­peal more to the people who already sup­port him. That’s how you might build an audi­ence for an “alt-right” me­dia com­pany.

Maybe. There’s an­oth­er per­fectly plaus­ible ex­plan­a­tion, however: Trump, and the ad­visors he re­cently pro­moted, genu­inely buy in­to the idea that “let Trump be Trump” is the strategy that won the primar­ies and is there­fore their best chance for the gen­er­al elec­tion. That Trump will no longer have to chafe un­der the re­straints that Man­a­fort wanted to put on him is the icing on the cake. After all, so this line of think­ing goes, Man­a­fort was in charge for 2 months and Trump’s num­bers went down, not up.

A key piece of evid­ence that Gruber cites is this quote from a former Trump staffer, Stephanie Cekiel­ski:

Even Trump’s most trus­ted ad­visors didn’t ex­pect him to fare this well.

Al­most a year ago, re­cruited for my pub­lic re­la­tions and pub­lic policy ex­pert­ise, I sat in Trump Tower be­ing told that the goal was to get The Don­ald to poll in double di­gits and come in second in del­eg­ate count. That was it.

The Trump camp would have been sat­is­fied to see him polling at 12% and tak­ing second place to a can­did­ate who might hold 50%. His can­did­acy was a protest can­did­acy.

I’ll just point out that this ex­pect­a­tion — of Trump com­ing in second in the primar­ies — came from Trump’s ad­visors, not Trump. It’s likely that, a year ago, Trump’s ad­visors were just as per­plexed by his can­did­acy as the rest of us were. And al­though Ms. Cekiel­ski firmly be­lieves that Trump got in­to the race not ex­pect­ing to win (an opin­ion shared by Mi­chael Moore, claim­ing some un­spe­cified in­sider know­ledge1), she points out what is per­haps Trump’s most sa­li­ent per­son­al­ity trait:

The Don­ald does not fail. The Don­ald does not have any weak­ness.

As Nate Sil­ver points out, Don­ald Trump is doub­ling down on a los­ing strategy. Maybe that’s be­cause he’s sab­ot­aging a cam­paign that he nev­er ac­tu­ally wanted to win. Or maybe the truth is as ob­vi­ous as Mr. Trump’s hair­piece: Don­ald Trump is a self-ab­sorbed nar­ciss­ist who ac­tu­ally be­lieves that the best way to win the elec­tion is to put his per­son­al­ity on full dis­play in­stead of try­ing to smooth it over.

Let’s not for­get Han­lon’s2 razor: “Nev­er as­sume malice when stu­pid­ity will suf­fice.”


  1. It’s dif­fi­cult to ima­gine a scen­ario in which Mi­chael Moore, the second most vil­i­fied lib­er­al after Hil­lary Clin­ton, would be­come the con­fid­ant of a Trump and/or Re­pub­lic­an in­sider. Not im­possible but very dif­fi­cult. 

  2. Or is it Hein­lein’s razor

Permalink


Django Print SQL Middleware

Django’s ob­ject-re­la­tion­al map­per (ORM) is per­haps that frame­work’s best fea­ture. It’s both easy to use and very power­ful. Nev­er­the­less, I be­lieve it’s im­port­ant to keep an eye on the quer­ies that it gen­er­ates. It’s of­ten easy to miss cases where you’ve un­in­ten­tion­ally triggered a new query when ac­cess­ing a for­eign key or when du­plic­ate quer­ies are be­ing gen­er­ated.

The Django De­bug Tool­bar is prob­ably the most widely used tool for peer­ing in­to Django’s data­base activ­ity. Per­son­ally, though, I’ve al­ways found that it tends to get in the way of the UI that I’m de­vel­op­ing. I’ve long pre­ferred to see the SQL out­put to the con­sole in­stead. I used to use Django devserv­er’s SQLRe­al­TimeModule for that pur­pose but devserv­er has been run­ning in­to com­pat­ib­il­ity prob­lems with re­cent ver­sions of Django. So, I’ve writ­ten my own mid­dle­ware based on the devserv­er mod­ule. I re­cently up­dated it to work with Django 1.10’s new style of mid­dle­ware, so I thought I’d share it:

import re
from decimal import Decimal

import sqlparse
from django.db import connection

_sql_fields_re = re.compile(r'SELECT .*? FROM')
_sql_aggregates_re = re.compile(r'SELECT .*?(COUNT|SUM|AVERAGE|MIN|MAX).*? FROM')


def truncate_sql(sql, aggregates=True):
    if not aggregates and _sql_aggregates_re.match(sql):
        return sql
    return _sql_fields_re.sub('SELECT ... FROM', sql)


def printsql_middleware(get_response):
    def middleware(request):
        response = get_response(request)
        num_queries = 0
        time = Decimal('0.0000')
        for query in connection.queries:
            num_queries += 1
            time += Decimal(query['time'])
            sql = truncate_sql(query['sql'], aggregates=False)
            formatted_sql = sqlparse.format(sql, reindent=True)
            for index, line in enumerate(formatted_sql.split('\n')):
                if index == 0:
                    print('[SQL] {}'.format(line))
                else:
                    print('      {}'.format(line))
        deduped = set([q['sql'] for q in connection.queries])
        print('[SQL] {} queries, {} duplicates. {} seconds.'.format(num_queries, num_queries - len(deduped), time))
        return response
    return middleware

Of course, you should only use this mid­dle­ware in de­vel­op­ment and nev­er in pro­duc­tion. It re­quires SQL­Parse to be in­stalled.


“PyPy Gets Funding From Mozilla for Python 3.5 Support” →

Moz­illa re­cently de­cided to award $200,000 to PyPy as part of its Moz­illa Open Source Sup­port (MOSS) ini­ti­at­ive. This money will be used to im­ple­ment the Py­thon 3.5 fea­tures in PyPy. With­in the next year, we plan to use the money to pay four core PyPy de­velopers half-time to work on the miss­ing fea­tures and on some of the big per­form­ance and cpy­ext is­sues.

This is good news in­deed. I’ve been us­ing Py­thon 3.x in all new pro­jects for some time now. For a while, I was us­ing pypy3 in those pro­jects un­til my most fre­quently used lib­rary, Django, dropped sup­port for Py­thon 3.3 (which is the ver­sion that pypy3 cur­rently tar­gets). I’m look­ing for­ward to re­gain­ing the per­form­ance ad­vant­ages of us­ing PyPy.

Kudos to Moz­illa for sup­port­ing open source soft­ware de­vel­op­ment. Hope­fully oth­er cor­por­a­tions and or­gan­iz­a­tions will fol­low their lead.

Permalink


First Private Mission to the Moon →

The first com­pany to ap­ply for a com­mer­cial space mis­sion bey­ond Earth or­bit has just re­ceived ap­prov­al from the fed­er­al gov­ern­ment. As part of the Google Lun­ar X Prize com­pet­i­tion, Moon Ex­press in­tends to launch a small, single-stage space­craft to land on the Moon by the end of 2017.

I’m very ex­cited to see if any­one can claim the Lun­ar X Prize be­fore the end-of-2017 dead­line.

Permalink


Frequent Password Changes Are a Bad Idea →

Cran­or even­tu­ally ap­proached the chief in­form­a­tion of­ficer and the chief in­form­a­tion se­cur­ity of­ficer for the FTC and told them what a grow­ing num­ber of se­cur­ity ex­perts have come to be­lieve. Fre­quent pass­word changes do little to im­prove se­cur­ity and very pos­sibly make se­cur­ity worse by en­cour­aging the use of pass­words that are more sus­cept­ible to crack­ing.

Over the past few years, or­gan­iz­a­tions in­clud­ing the Na­tion­al In­sti­tute of Stand­ards and Tech­no­logy in the US and UK gov­ern­ment agency CESG have also con­cluded that man­dated pass­word changes are of­ten in­ef­fect­ive or coun­ter­pro­duct­ive.

Bingo.

Permalink


“When the Strong Outbreed the Weak” →

An ex­per­i­ment in se­lect­ive breed­ing for a de­sir­able trait shows that the simple ap­proach does not pro­duce the ex­pec­ted out­come:

The pur­pose of the ex­per­i­ment was to in­crease the egg-lay­ing pro­ductiv­ity of hens. The hens were housed in cages with nine hens per cage. Very simply, the most pro­duct­ive hen from each cage was se­lec­ted to breed the next gen­er­a­tion of hens.

If egg-lay­ing pro­ductiv­ity is a her­it­able trait, then the ex­per­i­ment should pro­duce a strain of bet­ter egg lay­ers, but that’s not what happened. In­stead, the ex­per­i­ment pro­duced a strain of hy­per-ag­gress­ive hens[…]. There are only three hens [left in the fi­nal cage] be­cause the oth­er six were murdered and the sur­viv­ors have plucked each oth­er in their in­cess­ant at­tacks. Egg pro­ductiv­ity plummeted, even though the best egg-lay­ers had been se­lec­ted each and every gen­er­a­tion.

The reas­on for this per­verse out­come is easy to un­der­stand, at least in ret­ro­spect. The most pro­duct­ive hen in each cage was the biggest bully, who achieved her pro­ductiv­ity by sup­press­ing the pro­ductiv­ity of the oth­er hens. Bul­ly­ing be­ha­vi­or is a her­it­able trait, and sev­er­al gen­er­a­tions were suf­fi­cient to pro­duce a strain of psy­cho­paths.

There are many, many in­ter­est­ing things to learn here. Most dir­ectly, this is a les­son in just how com­plex bio­logy is. But there may also be les­sons here for hu­man so­ci­ety:

In a par­al­lel ex­per­i­ment, Muir mon­itored the pro­ductiv­ity of the cages and se­lec­ted all of the hens from the best cages to breed the next gen­er­a­tion of hens. […] Egg pro­ductiv­ity in­creased 160% in only a few gen­er­a­tions, an al­most un­heard of re­sponse to ar­ti­fi­cial se­lec­tion in an­im­al breed­ing ex­per­i­ments.

It’s easy to cred­it the dir­ect­or with the suc­cess of a great movie. But have have you ever watched the en­tire list of cred­its at the end of your fa­vor­ite movie? Mod­ern movies do not spring, fully formed, out of the mind of the dir­ect­or, as Athena from Zeus. Nor are they simple col­lab­or­a­tions among the dir­ect­or and act­ors. They are the products of small armies of people, work­ing col­lect­ively to pro­duce something far great­er than any one of them could achieve on his or her own. The same is true of any great achieve­ment.

Permalink