How many described species are sampled in published phylogenetic trees?

You may have high expectations for this post! Don’t – answering this question in a serious way would take a lot of research. Instead I’m just scrawling a few notes here for reference. Maybe someone else can do the legwork.

I was asked this question and came up with a quick estimate of 10%. Here’s what went into this number (admittedly somewhat post hoc):

The Open Tree project’s phylesystem – a repository of curated phylogenetic trees – samples about 2% of the species that are listed in the taxonomy (OTT) which has 2.2 million tips. That would be about 45,000 species. I’m going from memory here; I did this measurement several years ago and could remember incorrectly. So you’d would have to double-check that. The number might be in one of the Open Tree papers, but it’s easy to compute by traversing the JSON files in phylesystem and checking them against the taxonomy (filtering by rank if you want to be picky).

2.2 million could be high (due to species names with unusably bad descriptions, undetected synonymies, and so on) or low (due to published species not finding their way to any of the inputs to the taxonomy). I won’t touch it for now.

A lot of the studies that are in phylesystem are stored but not curated very well – so we may not have detected all the species that we have in a computationally useable way. Let’s raise the estimate from 2% to 3% to account for that.

Phylesystem has trees for maybe 6,000 published studies (5,000 to 8,000 – memory again). How much of the literature does this cover? I’m sure there are published estimates of the number of published phylogenetic studies but the references aren’t at my fingertips. We know phylesystem is quite incomplete (Drew et al), so 10,000 studies has to be a lower bound. 20,000 seems plausible assuming we’re doing some filtering (e.g. counting only studies that sample at least 3 described biological species – not linguistic phylogenies and so on). I think I’ve heard an estimate of 50,000 – I don’t remember where – and that seems plausible too.

With each additional study, there will be diminishing returns in that the species sampled may also be sampled by other studies. But let’s ignore this, it’s too much complexity for such a crude estimation.

In addition, the number of species per study is clearly not constant. Open Tree may have selectively chosen large or ‘high-yield’ studies for curation, leaving fewer species left to occur in unselected studies. I will ignore this complication too.

So, assuming 3% coverage for every 6,000 published studies, and 20,000 published studies, that would be 10%.

Categories: Uncategorized

She is an adjective

Geoff, you’ve closed comments (in spite of inviting them) so I can’t reply on your post, so I’ll reply here. You claim to be stymied by this assertion:

Jane Jacobs has become more than a person. She is an adjective.

You made barely any effort to find examples or an interpretation for this. A web search for “a Jane Jacobs” (designed to elicit adjectival uses) turns up plenty of examples of “Jane Jacobs” used as an adjective, so the evidence that you linguists are so fond of is out there. Probably the above refers to “a Jane Jacobs walk” which evidently is a kind of walk or stroll. For example, I find: “Anyone can host a Jane Jacobs Walk.”

Yes, “Jane Jacobs” is not written with quote marks in “Jane Jacobs … is an adjective”, and I am fully on board with the idea that a person cannot be an adjective. But it is common in situations like the above to play around with the referential transparency a bit, for the sake of levity.

I haven’t written a grammar of English like you have. Perhaps “Jane Jacobs” in “a Jane Jacobs walk” does not qualify as an adjective? This would surprise me, but regardless, it seems to me that telling the original writer that “Jane Jacobs” is not an adjective in “a Jane Jacobs walk” would be petty.

UPDATE 10 July: HST informs me that the construction “Jane Jacobs walk” is a noun-noun formation and not an adjective-noun formation. But this is not the issue. I am talking about Geoff’s statement

I have absolutely no idea what the blurb-writer could have meant

Really? Is Geoff so unresourceful that he could not have come up with the blurb-writer’s intended meaning (which I found to be pretty easy to hunt down, and quite clear)? Or is he just pretending not to know, so that he can punish the blurb-writer and all the rest of us for stretching the word “adjective” beyond its correct technical meaning to include other kinds of noun modifications? I applaud Geoff for giving Strunk and White such a hard time over their use of words, but this case is hardly similar: first, his examples from Strunk and White may not admit a coherent non-technical interpretation (although I have not tried to discern one for their many uses of “passive voice”), and second, Strunk and White have (or had) an obligation to use technical terms properly, because they are writing technically about the technicalities of grammar and usage. The blurb-writer is under no such obligation and has a right to employ whatever colorful rhetorical devices  he/she likes, as long as the result is comprehensible and true.

Categories: Uncategorized

Apple confuses deputy

The MacOS API has something called an NSURLSession object, which relies on a background process (daemon) called ‘nsurlsessiond’ (if I understand correctly). If an application wants to fetch a web page, it uses an NSURLSession object, which pokes the daemon, which pokes the site, which returns the bits to the daemon, which returns the bits to the application.

Because I’m curious and a bit paranoid, I use a program called Little Snitch, which monitors all network requests made by applications. Little Snitch implements a conventional access control system: you can grant or deny any application access to network hosts, ports, etc. If you don’t want, say, gamed making any outward network connection, you can make a Little Snitch rule that prevents that.

So what if a program uses nsurlsessiond to make a connection? Little snitch only knows that nsurlsessiond is asking for network access, so there is no way for it to grant different permissions to the different programs that are _using_ nsurlsessiond. I can’t ask Little Snitch to let, say, icloud make connections, but not to let gamed make connections, because Little Snitch only knows from nsurlsessiond, not gamed or icloud. So I have to allow neither or both.

This is a classic confused deputy scenario. I have to let nsurlsessiond connections through, or I can’t get work done. But when I do so, some evil principals will gain access I didn’t want them to get.

I never noticed nsurlsessiond before a recent upgrade. It’s not a bad programming pattern; a reuseable piece of code is packaged up for general use. In fact deputies (abstractions, services) of this kind are generally considered good practice. The evil comes when use of a deputy ‘launders’ access so that information needed for permission-granting is lost. This particular deputy has a daemon, I guess so that requests from many programs can be coordinated, with the side effect that requests are ‘laundered’.

It may sound like I’m suggesting that the whole authority chain should be preserved through to the point where an access decision is made, so that Little Snitch can see it, but in general that doesn’t work either. There are ways to do this right (capability architecture), but they usually require an overhaul of the code, and a rewrite of the operating system…

Categories: Uncategorized

Oldest US entomological societies?

To: (contact address on site)

The Entomological Society of Pennsylvania was founded in 1842, yet the American Entomological Society says that the AES, founded 1859, is the “oldest continuously-operating entomological society in the Western Hemisphere.”

What is the explanation of the difference? Did Ent. Soc. of PA suspend operations at some point, or is AES wrong?

There seems to be a lot of confusion about early American entomological societies. A recent article at the Biodiversity Heritage Library blog says that the AES is the oldest entomological society in the U.S., without qualification. This is clearly wrong. (They also neglect the Cambridge Entomological Club, “operating continuously” since 1874, in claiming that NYES was perhaps number three.) I intend to send them a correction.

(I too was guilty at one point of giving out incorrect information about early entomological societies. I’m not sure even now that I know what the first four were.)

Jonathan Rees
former treasurer of the Cambridge Entomological Club
“The first truly entomological society in America was the
the Entomological Society of Pennsylvania formed in 1842”
“The American Entomological Society is the oldest
continuously-operating entomological society in the Western Hemisphere, founded on March 1, 1859.”
“Depending on how you count, the New York Entomological Society (NYES), founded in 1892, is either the second or third oldest entomological society in the U.S. The oldest is the American Entomological Society, founded in 1859 in Philadelphia; the Brooklyn Entomological Society was founded in 1872, but merged with the NYES in 1968.”
“Entomological societies which preceded ours and which have continued to publish regularly are: 1) The American Entomological Society, 1867, …”

[Update 2016-09-29: The ESP got back to me with the following: “The ESP was founded in 1842, but fizzled in 1844. After a brief 80 year hiatus, it was started back up again in 1924.”]

Categories: Uncategorized

Digital preservation and independently held copies

[2016-01-25 Title updated to be more specific.]

Some information – writing, data, images – is important enough that it should be preserved and made available for as long as possible. Somebody, 5 or 10 or 50 or 200 years form now, might want or need to look at it. If you care that something be preserved, you will ask yourself what you can do to help bring about preservation.

It’s very easy for an individual, a project, or an organization to say: I am in control of this information, I am a responsible member of the community, and I can be a good steward. I will use the best redundancy technology and keep good backups, so the stuff will be safe from fire, natural disaster, and so on. It will be preserved because I will preserve it. (See e.g. NARA’s codification of being responsible.)

This may be true, up to a point, but it is a delusion. The risk that an individual, project, or organization might suddenly lose its ability to preserve is too great, in my opinion, for this to be an acceptable digital preservation solution by itself. Individuals die or become disabled; projects get canceled by management under budget pressure or changes in priorities; and organizations close or go bankrupt. And everyone is vulnerable to legal and governmental takedowns and censorship, and acts of war. These are all very unlikely events, but over long periods of time, unlikely risks become somewhat likely.

Every preservation plan must therefore include distribution of the information to one or more independent parties that are very likely to survive threats against the original steward. The receiving parties should be organizationally and legally independent of the original steward, and should reside in a different jurisdiction (country). They should keep their copy because they want to, not because they are being paid to.

Someone who gets one of these copies should by ready, if necessary, to make it available for use and perhaps further dissemination and preservation planning.

This is whether we’re talking about Very Important Stuff handled by big well-funded entities, or stuff that’s extremely informal and small-scale. If it’s useful in your community, make sure a friend in another country has a copy.

Oddly, this problem used to be solved, but is now unsolved. During the print era, the natural and economical way to disseminate information was to make lots of copies and get libraries to take them up. Redundancy was a completely natural side effect of copying technology and economics. The Internet works in a completely different way: copies are made on demand (copied from the server to the client) and thrown away. There are content distribution networks (CDNs), but these are ephemeral and dependent (under contractual control of the original steward). We no longer have independent stewards of copies of things because we don’t need to to support our day to day habits.

(If the stuff in question is an active database, the recipient may also choose to continue updating it, or give it to someone else for update coordination, but this is an optional and orthogonal secondary step. The main point is that the information should be preserved, because someone might need to know what it says.)

If the “backup” is to become the new principal steward – and one should always be prepared for this – it will be important to transfer domain names as well. If the original steward is incapacitated, then the backup organization will have to change the DNS records without coordination with the original. That means prior transmission of registrar passwords. Arrangements like these are complicated and fragile, and therefore much rarer than they need to be. An excellent example of organizations doing the right thing in this regard is the coordination between FOAF and Dublin Core.

I was telling this story around 2007 to anyone who would listen, as part of my work for Science Commons. One of the most important infrastructure databases for scholarship is the Crossref DOI metadata – the information that gives you basic bibliographic information for the publication associated with a DOI. At the time I didn’t know whether Crossref was copying its database to an independent foreign partner, and maybe it wasn’t, but by 2010 Crossref had announced backup to Portico, which sounds pretty good to me – Crossref is a UK organization, Portico is a US organization, and neither would be made vulnerable by the other’s legal or financial trouble. The fact that Crossref issued a press release about this tells me that the idea of independent copies is neither obvious nor silly.

Twitter is not a very good way to carry on a conversation, but it has the advantage of being public, which helps keep people honest and responsive. ORCID is a fairly new organization that has an infrastructure database similar to Crossref’s, one that is starting to gain an important role in scholarship. On 18 January I casually asked:

wondering, does @orcid do outside-org outside-country backups like @crossref does ( …)?

The answer from @ORCID_org:

@jar346 @orcid @crossref Yes, we have backup servers in countries outside of US.

This didn’t answer my question; to me a “backup server” is something administered by the originating organization, perhaps physically residing in a different locale but not necessarily accessible to any “outside-org” there. And I found nothing on their site to reassure. Rather than continue on twitter I wrote this post. Maybe they will read it and get a better idea of what I was trying to say.

Don’t get me started on copyright.

Categories: Uncategorized

My Quora experiment

Generally I stay away from Quora because of all the inanity there, but I keep going back because there’s just enough good stuff (e.g. Keith Winstein keeps posting there).

After reading one of Philip Greenspun‘s blog posts (I forget which one) I got to thinking about public education. Two peculiar things about it are (a) we pay to send other people’s children to school, even though education seems to be a private benefit (certainly college is considered to be one), (b) we make it illegal for a parent not to. (a) is simply liberal, once you see that education is a public good, not a private one, so not really more puzzling than public investment in roads. But (b) requires some justification since on the surface it sounds like meddling in personal liberty, as well as unnecessary since isn’t education in one’s self interest?

I did some web searches around (b) and didn’t turn up very much. Mostly discussions of public education go to (a), talking about all the benefits of an educated public, and don’t address (b). The best reason I found was that forcing parents to send their kids to school protects the children since it keeps the children from being exploited for their labor in factories, on farms, and so on. Self-interest is not a good evaluation heuristic here because the parents’ interest may be at odds with the child’s interest.

There was also something about integrating the children of immigrants.

Maybe there is so little dissent from compulsory education that nobody questions it. You don’t see picket lines with people shouting “no more education”. As Philip would say, parents like the free day care.

My pet rationale for compulsory education is that it is defensive: children grow up to be voters and jurors, and when we are falsely accused we don’t want to be judged by the ignorant. We have to coerce people to be less ignorant, since otherwise they would choose to be ignorant. That is just a theory. Maybe school helps enlighten students, but public opinion polls would suggest it’s not very successful at it.

I want to emphasize that I’m not being polemical; I’m not asking the question because I have an axe to grind about how children ought to be free to skip school and parents have no responsibility if they do. I’m just looking for an answer to what I thought was an obvious question of political philosophy, and a relatively uncontroversial one given that you don’t hear a lot of fighting about it.

From time to time you do hear people complain about paying property taxes when they don’t have children or when their own children don’t benefit from the local public schools, and it would be nice to have a sensible answer to such complaints.

So I tried Quora. The way I asked was: “Why do we require, and pay, other people to send their children to school?” There were three serious flaws with this way of asking, and as a result the exercise was unproductive.

First, it is two questions; requiring and paying are very different things, as I say above, and they have different rationales. Most of the answers addressed the ‘paying’ part while completely ignoring the ‘requiring’ part. I’ve found something similar with email: if you ask two questions in an email message, the response(s) you get back will invariably answer one or the other but not both. If you have two questions, send two messages.

Second, it does not make clear that I was looking for a rationale for requirement that would decisively overcome the liberty argument.

Third, it talks about sending children to school, when it should be asking about compulsory education – home schooling is perfectly OK. So I got an answer picking at this flaw in the question, without giving me any response to the ‘requiring’ part.

I hope my report of these missteps will be of help to someone else formulating a question for Quora or any similar forum. A better question would have been: “What gives us a moral right to tell others that their children have to get an education?” – that actually helps generate hypotheses, such as uneducated = dangerous (making it similar to the imposition of building codes).

What useful information did I get? Here are excerpts (I am quoting people out of context, go back to for justice to them):

“The more educated people there are in your world, the larger your pool of potential good friends will be and the more interesting your life will be.” – this goes to (a), not (b).

“Because the collective cost of ignorance to society is far, far more expensive.” – this says why you would want to require education, not why you would have a right to do so.

“Requiring kids to go to school isn’t the only way kids can be educated.” – as I described above, the purpose of this response was to (justly) put in a plug for home schooling.

All the other answers were about benefit to society and why the public pays for education. No quarrel there. One responder taught me the term “merit good”, which was nice.

Did I learn anything? Yes, about how not to post questions to Quora, but not about the question at hand.

[addendum 2016-06-07: This Language Log post contains the kind of information I was looking for: James Garfield in his 1881 inaugural address said “All the constitutional power of the nation and of the States and all the volunteer forces of the people should be surrendered to meet this danger by the savory influence of universal education.” referring the danger that illiteracy poses to the survival of the republic. That is, he says it’s not just a Good Thing, it’s a matter of addressing an existential threat, and therefore necessary.]

Categories: Uncategorized

Why is Open Tree not publishing RDF?

Question raised on an Open Tree discussion group:

I’m wondering why you are not using RDF as the underlying graph data model and OWL annotations (and other existing ontologies) to create a semantic graph and therefore following the current best practices to build knowledge graphs.

Good question. Partly it’s that only one person on the project knows anything about RDF. But I think this is mainly a matter of cognitive space and time among the developers, and priorities. If we felt a need to do it given the goals that we have, we would probably do it. But we haven’t felt any need.

Converting to RDF and OWL is easy to do poorly (and perhaps adequately for many purposes). One of the first things I did on the project was to convert the taxonomy to turtle so I could load it into a triple store. (I was on the RDF bandwagon for many years.) Anyone could do this; it’s a trivial script. Also, the NeXML format that we use subsumes RDFa Core so can be converted easily – in a sense we *do* publish RDF for the study database.

Doing RDF/OWL well is much harder, and would require cooperation with other groups such as OBO (IAO, VTO, …) and TDWG, choice of and support for persistent URLs, good term definitions and documentation, SPARQL endpoint, and so on. These coordination activities are extremely time consuming. Of course doing so would be lovely in the abstract, but there has been no reason for us to make this a priority.

In my experience, format conversion is by far the easiest activity in data ecology, so mere conversion to RDF has little value. The hard parts are marshalling the data in the first place, and then using it wisely. Due to the vagueness of most vocabulary term definitions, the best laid RDF usually requires as much reverse engineering and postprocessing as data in any other format when doing data integration and analysis. So it is semantics, not syntax, where the effort is best spent. (RDF being a syntactic play, and not helping with semantics any better than any other data format, in spite of the buzzword “semantic web”. OWL helps semantics a little but only with inference, not with ground truth, which is what really matters.)

The feedback captured in the feedback system (in github) has a little structure, and we could probably do better in obtaining more.

The thing that would tip the balance would be a real funded collaboration with another project where there was good reason to use RDF or OWL for communication between the collaborators. Publishing RDF/OWL merely for the sake of doing so is not in my opinion the best use of resources – especially given that all the information is open and anyone else could do such a conversion for us. I read a lot about the size of the linked data cloud, but very little about its utility. I bet there are legitimate uses of RDF-published data, but from what I’ve seen people mostly publish RDF just so that they can say that they did, not because they know that someone needs it. (Would love to be shown otherwise.)

How would having RDF for open tree make a difference to you, personally?

Categories: Uncategorized