Aol dials up the stupid with 150 editorial layoffs

Aol dials up the stupid with 150 editorial layoffs

The ethics of reporting on the Sony hack

The ethics of reporting on the Sony hack

Perverting the Metric: The Role of Metrics in Editorial Strategy

HuffPo and BuzzFeed co-founder Jonah Peretti recently said in a long and fascinating interview by Felix Salmon published at Matter:

I love metrics and I love thinking about optimization, but I think that the optimal state is being slightly suboptimal because as soon as you try to actually optimize, particularly for a single metric, you end up finding that the best way to optimize for that metric ends up perverting the metric and making the metric mean the opposite of what it used to mean.

This reminded me of an idea I’ve been kicking around for a while about how best to approach digital editorial strategy: it requires an ability to wield metrics, vision and instinct in just the right proportions.

It’s something I’ve been a part of for my own tiny blog here, an arts and culture website I co-founded, and even a business journal’s web presence. I’ve learned a few important things from my experience with editorial strategy, and while none of them are particularly surprising or mysterious, I think writing them out will be helpful to myself and perhaps to others.

Contribute to the conversation

Metrics are a great place to begin a conversation about editorial strategy but a terrible way to end it. I’ve seen metrics substituted for thinking critically about editorial direction all over the web, and what’s worse is I’ve been in the room when some of those poor decisions were made and I failed to object. It’s not a mistake I’m proud of, nor one I would make again.

But it’s easy to criticize after the fact. True leadership demands urgency. Whenever metrics are the deciding factor in an editorial decision, someone is making a mistake and it’s your responsibility to tell them.

Be respectful when their name is closer to the top of the org-chart than yours, but be direct and back up your assertions with evidence. Even if you’re outranked by everyone else in the room, at worst, you’ll be ignored, and at best you’ll show initiative and concern for the publication’s success.

I’m not saying there is no place for metrics in editorial strategy. They should absolutely be involved in the decision-making process, but they should never be the sole ingredient. In other words, these days metrics are usually necessary1 but never sufficient to make an informed editorial judgement.

Reactive vs. critical thinking

Pure reactivity is the wrong way to use metrics, and looks something like this:

“Everyone clicks this type of story, so let’s do more of this type of story!"

Don’t use metrics to narrowly define editorial strategy. After all, an algorithm could do that with little or no human intervention (and, as I’ll discuss below, they often do). Popular topics don’t need much additional promotion. They surface organically and allow you to focus on promoting lesser-known work of equal quality. This is a powerful concept if you’re wiling to use it in your strategy sessions.

Use metrics as one factor in your strategy. After all, the numbers are way to read between your own lines and to learn what drives popular content beyond mere keywords. That looks something like this:

“Everyone clicks on this type of story. What about it, beyond the mere subject matter, makes it so appealing?"

One problem, many possible solutions

There are many reasons some content does more pageviews, higher time-on-page or lower bounce rates than other content. Here are some illustrations of the problem of a narrow band of popular topics getting the majority of attention, and some ways I have thought up and in some cases successfully implemented to solve the problem.

The “Top Post” Filter Bubble

Eli Pariser popularized the idea of the filter bubble, an explanation for how tailored web content reinforces viewpoints with which we already agree, and insulates us from alternative perspectives. Metrics are often used to do this on websites.

The most-read stories of the previous day might be featured prominently in the sidebar. This additional exposure gets them even more clicks, and even if the software causes articles older than one day to “age out” of the featured-posts box, it still severely limits the potential for featuring other articles.

This may be the problem at some sites: your digital publication doesn’t know how to surface its best content. Consider adding to popular posts some links to less popular but equally valuable content. This will combat the filter bubble and help expose readers to good stuff they may otherwise miss.

The Slideshow Site

Slideshows are a dangerous game. They are almost guaranteed to turn your steady daily traffic into a big spike. If even half your daily visitors go through even half a 20-slide show, you’re doing five times your usual traffic that day. If you’re not careful, you risk becoming known as the slideshow site, instead of the news site.

If you insist on building slideshows, use myriad internal links to point your slideshow viewers to your substantive content. Better yet, work with in-house or outside developers to automate internal links to archive pages. For example, if you run a site about New York, the first time the name “Michael Bloomberg” appears in an article, your content management system could auto-generate a link to a page listing all articles mentioning his name.

10 Things About Headlines You Have to Read to Believe

Sorry to mislead you, but I’m only to going to talk about one. Slideshows often have numbers in the headline by definition. That is one explanation for why they’re so popular. People like headlines with numbers, as a quick search for “numbers in headlines” will illustrate.

I don’t advocate making every article a list. In fact, that’s a terrible idea, at least for news sites. But it’s worth incorporating numbers into headlines where it doesn’t look forced. For example, instead of “CEOs cite multiple syngeries as key to upcoming merger,” try “3 reasons Hospital 1 and Hospital 2 are merging, straight from the CEOs.”

On-point but out of sight

Maybe topics clearly within your site’s wheelhouse don’t perform well, no matter how many headlines, reporters or A/B tested tweets you use to produce and market them. This may simply mean the audience for those topics is substantially smaller than your broader audience. Don’t wait for the audience to find you.

I had great success finding an audience for some very niche stories because I sought it out on Reddit, in web forums, in Google+ Communities, with Twitter hashtags and more. The idea is that there are groups of people who self-select for interest in topics otherwise lacking broad appeal. Those audiences are smaller, but they are also more engaged, so the time spent finding them is worth it.

These are just examples, and the problems differ from site to site. But I think they explain the value and the limits of metrics in evaluating and improving editorial strategy at digital publications.


  1. If I say metrics are always necessary to make an informed editorial judgement, I omit the occasionally successful-despite-what-the-metrics-suggest, good-old-fashioned gut decision, and I’m not comfortable doing that. 

Journalism and tomatoes

Journalism and tomatoes

The true Julian Assange

The true Julian Assange

Om Malik on digital advertising

Om Malik on digital advertising

Down the aggregation rabbit hole

This began as a link post pointing to Joel Achenbach’s Washington Post blog entry Journalism is aggregation. But, like more and more link posts lately, it got away from me and merged into its own article.

Achenbach decides journalism is aggregation, and that’s okay. Or maybe he decides it’s not really aggregation, or that it’s at an acceptable point along the continuum between valueless aggregation and value-adding aggregation. I’m not actually sure he settled on a conclusion, and that’s okay, too.

Achenbach had reminded me of my recent post on the topic, which prompted the author of the post I was commenting about to leave me a nice comment of his own. (So meta!)

And then I found this post by Joshua Benton at the Nieman Journalism Lab expressing some disappointment with Achenbach’s aforementioned lack of a conclusion, or more precisely paints his conclusion as a bit of backtracking. I’m not sure I agree with Benton’s take, but both men raise interesting points.

Some journalism is aggregation, but most good and all great journalism is more than mere aggregation. It’s a synthesis of the anecdotes, data, facts and perspectives of as many reliable sources as you can fit into your word limit. Right? That’s an accurate description of much of the best journalism I’ve read in the past year or two.

To me, pure aggregation on the web involves sharing a link and perhaps pairing it with an inflammatory or vapid comment. Sometimes, that’s actually fine with me. Twitter is a good example of that. But no one would call it journalism, even when journalists do it. That’s important to keep in mind, I think: not everything a journalist does online or off is, or is intended as, journalism.

Achenbach wrote about the interview process, which most journalists use to offer different perspectives on a piece of news from experts of different disciplines or schools of thought. Interviews, collecting the commentary of multiple sources, are an aggregation of those opinions. But that’s just one activity in the composite of activities which together compose an act of “journalism.” The result is an article in which, as Achenbach also says near the end of his post, the reporter has drawn on personal knowledge, research and experience beyond the interviews and facts aggregated.

Aggregation, then, is the collection and presentation of opinions or facts, adding little or no context. Journalism, I think, is an equation like this:

Journalism = aggregation + context

I want to know what you would add to, or remove from, that equation. I realize that aggregation on the web generally, and as an issue in digital journalism and publishing specifically, predates my humble blog posts on the topic, but I want to explore it with other interested folks.

Margaret Sullivan takes her NYT colleagues to task like it's her job, because it is

Margaret Sullivan takes her NYT colleagues to task like it’s her job, because it is

Jay Rosen on the "View from Somewhere"

Jay Rosen on the “View from Somewhere”

Is Google Play Newsstand a viable alternative to standalone Android apps?

A new kind of freelance journalism

A new kind of freelance journalism

FBI asks DOJ to investigate source of Calderon leak to Al Jazeera

FBI asks DOJ to investigate source of Calderon leak to Al Jazeera

Jim Roberts joins Mashable as executive editor, chief content officer

Jim Roberts joins Mashable as executive editor, chief content officer

Reuters nixes Next: Failed redesigns and the challenge of expanding a digital audience

Reuters nixes Next: Failed redesigns and the challenge of expanding a digital audience

NYT managing editor: Guardian story on Israel and N.S.A. Is Not 'Surprising' Enough to Cover

NYT managing editor: Guardian story on Israel and N.S.A. Is Not ‘Surprising’ Enough to Cover

Daniel Victor of the New York Times shows us how to be a reporter even on Twitter

Daniel Victor of the New York Times shows us how to be a reporter even on Twitter

How Brown Moses exposed Syrian arms trafficking from his front room

How Brown Moses exposed Syrian arms trafficking from his front room

Legislative failure to define essential terms

Legislative failure to define essential terms

Strongbox and Aaron Swartz: Open source, anonymous tips

Strongbox and Aaron Swartz: Open source, anonymous tips

Philadelphia Inquirer, Daily News splitting into separate paywall sites

Philadelphia Inquirer, Daily News splitting into separate paywall sites

Washington Post reports "a person familiar with the plans" says Washington Post considering paywall

Philly.com posts NSFW photo, ‘production glitch’ blamed

Brazilian newspapers leave Google News en masse

Brazilian newspapers leave Google News en masse

Welcome to VentureBeat’s reporting-driven Friday

Welcome to VentureBeat’s reporting-driven Friday

NYT quote approval policy is (only) a good start

The new quote approval policy at The New York Times, as quoted by Times opinion writer Margaret Sullivan:

So starting now, we want to draw a clear line on this. Citing Times policy, reporters should say no if a source demands, as a condition of an interview, that quotes be submitted afterward to the source or a press aide to review, approve or edit.

I first wrote about the quote approval problem when I linked to David Carr’s piece on it. Then I expressed my agreement with Temple Law professor David Hoffman, who wrote at Concurring Opinions about the frequency with which experts such as himself are misquoted or taken out of context.

I’m not sure the Times policy does a very good job of distinguishing between approval by PR folks and approval by subject-matter experts. The former try to approve quotes to control messaging, while the latter try to approve quotes to ensure their opinions on a given issue aren’t manipulated to further a skewed narrative.

I don’t think those two cases can be dealt with in the same policy without explicitly pointing them out and setting up a framework for each one. The Times policy allows for exceptions with senior editorial approval, and that may allow experts like Professor Hoffman to explain that they want to ensure their comments are presented in the manner in which they intend them to be presented. Or, it may not.

Marco Arment suggested disclosing when quotes have been approved for an article, instead of calling for an unqualified end to the practice. I’m not sure that’s the perfect solution, but I think I prefer Mr. Arment’s policy to the Times policy.

Disclosure makes sense and would show great respect to readers by allowing them to decide whether the reliability of a particular quote is or is not affected by its pre-approval by the source. Experts could ensure accurate representation of their opinions, and readers could be kept in the loop when a communications department has manufactured the CEO’s statement to the paper.

In short, the Times quote policy is nothing less than a good start, but it’s also nothing more.