Next steps for the new site

We’re now beginning work on the new version of the Science Blogging Aggregated site.

We’d like to have a working prototype of the site ready for the ScienceOnline conference in January.

Realistically, by then we’ll probably be able to implement the following features:

• Users login and register blogs
• Some sort of administrative check-off on registration, with anti-spam measures
• Aggregator compiles entries from registered blogs, displays on home page
• No tagging of individual posts, but blogs are categorized by user-specified “themes”
• Visitors can filter posts appearing on home page by theme

We may also add a language filter allowing users to specify their preferred languages. (This may be difficult to implement because it would require having curators in each language we support) Over the long term, we would like a multi-lingual interface, so all users can experience the site fully in their native language.

We are leaning towards a dense, information-rich layout for the home page, much like the existing home page, but with additional tools for users to filter posts, login, register, and so on.

In order to maximize the site’s utility, we are thinking about pre-populating the database. This would probably be a manual process, based on the existing feeds for ScienceBlogging.org. This would require an additional feature so that users could “claim” their blog and personalize their account. However, we’re not sure that’s doable by the January deadline. If readers can suggest models for how claiming a blog could work, with a minimum of fuss, we’d appreciate suggestions.

We are also considering a a new domain name for the site—we’d like it to be a truly notable name, one that’s memorable, says something about the site, and isn’t easily confused with some of the other science sites currently out there.

So here’s our plan for the next steps. We’ll keep you up to date as we continue to work on the project:

1. Develop a schema for a database that can handle the trimmed-down version of the site that we’re planning for January, but is flexible enough to meet our long-term goals
2. Arrange for site hosting. We can work on our existing personal server space for now but we’ll need a permanent home, and the sooner we find it the better.
3. Wireframe the first (limited-feature) version of the site: Create a template that developers can use to build the system, indicating what information will go on each page. Again, we may want to do this in anticipation of the higher-functionality site to come, so we don’t have to constantly reinvent the wheel.
4. Explore the process of creating a non-profit organization. This may be a larger non-profit that also includes ScienceOnline.
5. Create a schedule for the process of developing the site up through the conference.
6. Recruit additional help. We’re really short on programmers and designers. Any volunteers?

An outline for version 2.0 of the site

A few weeks ago I wrote up a tentative outline for the next generation of Science Blogging Aggregated. I’ve been sharing bits and pieces of it with you over the past week, but now I’d like to share the whole thing. It’s still a work in progress, a Google Doc that reflects our current thinking on the project—but of course, something that will continue to be refined as we move forward with the project.

Click here to view the document.

I’ve already tried to incorporate as many as possible comments from readers as I’ve shared the plans with you, but of course we continue to be open to additional suggestions. I think this is enough for us to use to get started, but there’s obviously much work still to be done. If you’d like to help out, you can either email us directly at contact@scienceblogging.org, or add a comment below and we’ll get in touch with you via the (hidden) email link you provide in the comment form. Particularly useful at this stage are people with CSS / web design experience, developers, and sysadmins.

We’re hoping to present a working prototype of the site at Science Online 2011. I’ve suggested a session on the conference wiki here.

We’ll continue to keep you posted and ask for your advice and suggestions as work progresses.

Building a better network: Identifying trends/posts of interest

When you build a network of blogging networks, the problem quickly escalates from “how do I collect as much data as possible?” to “how do I manage all this data?”

Take a look at the Science Blogging Aggregated home page. There’s lots of great stuff there — too much for the typical reader to handle. Even if you visit several times a day, the information rushes by too quickly to discern any trends, and it’s hard to know which posts are really well thought out and which are just one-off posts that hardly merit your attention at all.

We talked yesterday about one way of sorting through the data — tags. However, this method alone probably won’t satisfy all users. A person might be interested in all posts tagged “psychology,” but they might just want to see the highlights of what’s going on in other fields, and tagging won’t help them identify the most interesting, thoughtful posts.

We see at least four possible ways of sifting through the posts to find the most interesting ones.

1. Crowd-sourced ranking. Users rate or recommend posts they like, so others can sort by rating or number of recommendations to find the posts they want to see. An advantage is that there is no central authority telling readers what to like. A disadvantage is that blogs that are already very popular are perhaps most likely to be recommended, so this system might not help users identify up-and-coming blogs that are very high quality.

2. Self-promotion. Bloggers could promote a small number of their posts, indicating these are their best work (one per week? one per month?). This overcomes the “up-and-comer” problem, but a blogger whose work is mediocre could exploit the system by promoting posts that aren’t very interesting or useful to others.

3. Active curation. Editors could be chosen for each field (physics, biology, etc.) and actively promote one or two posts each day. That way readers would know that an expert has read all the posts on a topic and selected the most interesting or relevant. Advantages are that editors may be able to identify trends that more automated systems don’t catch, and that editors may be less swayed by the most popular blogs. Disadvantages include possible bias of editors, and variable editor quality. It would also require coming up with a system for selecting editors. Would a central person be in charge of that, or would we need to create some sort of a system for nominating/voting for editors?

4. Social networking. We could create a truly social network where users are only shown the “likes” of their friends. However, this requires a significant programming effort, and people are reluctant to join new social networks when they already participate actively in one or more networks. I think we might be better off using the social features of other networks, rather than building our own. If we could make it really easy for people to post their “likes” to Twitter and Facebook, then we could leverage those networks to perform the social function.

There is, of course, no reason that we shouldn’t do all of these things over the long run. But we have limited resources. Which of these approaches is most useful? Are there any other approaches that would work better? Do you have any specific suggestions for how to implement any of these ideas? Let us know in the comments.

Building a better aggregator: Goals, Tagging

The ScienceBlogging site you see now was always intended to be a temporary solution. What we really need is a site that not only aggregates blog posts, but also allows users to classify them, search them, highlight their favorites, point their friends to them, and do many other things we haven’t even imagined yet.

Behind the scenes, Bora, Anton, Jessica, Mark, and I have been discussing how to do that, but we realized that limiting the discussion to just ourselves is depriving us of a valuable resource: The people who’ll be using and contributing to the new site.

So, over the next few days, I’ll be offering some thoughts about how to proceed and inviting your comments. Our plan is to have at least a partially functional, working prototype of the new site by the ScienceOnline conference in January 2011. Let’s get that started right now by discussing the goals for the site.

Goals
Here are the goals we came up with for the site:

• To be a central site where scientists, media, other experts, and laypeople see what scientific topics are being discussed on blogs, in real time
• To be a resource for locating past discussions
• To promote science blogging and other online discussion of science
• To promote scientific accuracy and avoid pseudoscience and crackpottery
• To be encyclopedic and inclusive
• To be searchable and filterable
• To have a system (or multiple systems) for highlighting discussions and posts that are especially topical / high quality
• To have a means of removing or hiding posts that are not scientific (e.g. vacation photos, political rants unrelated to science, etc.)
• To be multilingual
• To be open source / open access

Should anything be added, changed, or removed?

Tags
One of the first considerations will be how to keep track of all this information, and a huge key to that will be classifying it. That’s why we think it will be essential to have a unified tagging system in place. If bloggers don’t select their primary tags from a central list, then it will be difficult for users to find posts on the topics that interest them. On the other hand, if bloggers must visit our site to choose primary categories, then usage will suffer. We can allow bloggers to set default tags for their posts using their registration page, but there should be some way to override those settings for individual posts, still using our list of preferred tags.

Could we create a WordPress plugin for this? Or adapt an existing plugin? What about other blogging platforms? What about templates that don’t support tags? One possibility is using a bookmarklet, which would be platform neutral but not ideal. Any other ideas on how to implement a tagging system?

That’s just the first bit — there’s a lot more to discuss, but we thought this would be a good way to get the conversation started. So please, let us know what you think in the comments.

Help us add images

Please let us know if it is possible – and how – to make a Feed of Flickr images with a particular tag. We would love to import photos tagged with #scienceblogging from Flickr.

Finding people to aggregate with

Aggregating blogs is not technically difficult. (See previous posts: How to create an aggregated feed and Feed aggregator choices.) Finding other people to aggregate with can be a challenge, however. Feel free to comment here in order to find other people to aggregate with. One way to start is to suggest a topic (neuroscience? medicine? new bloggers? meta-science blogging? students? faculty? physics? astronomy? anthropology?) that you blog about, and ask if others want to aggregate posts on that topic.

Remember, if you are an independent blogger who wants to be listed on scienceblogging.org, putting together an aggregated feed is currently the only way.

Also, remember that you can choose to aggregate only selected posts if you want, using tags as filters. Or you could aggregate your entire blog.

Feed aggregator choices

There are a bunch of feed aggregators out there. However, I haven’t used any of them, so I don’t know which ones work well and which ones work poorly. I encourage people to try them and comment here with their experiences. I can edit this post with useful information from the comments.

Web services

Below is a list of web services which will allow you to set up a feed aggregation.

• Yahoo Pipes. Widely used. Pipes has a blog where you can learn more about it. Some people seem to find it hard to use. Others complain that it edits the RSS feeds that it passes along (for example, changing whether the links open a new window or not). The Scienceblogs Diaspora Feed runs on Pipes; you can clone and edit that feed (this may be a good way to get started if you find Pipes confusing). There is a review of Pipes which might be interesting.
• FriendFeed. Another widely used one, and seems to be a better bet than Pipes (but comment here and say why or why not!). FieldOfScience uses this one. Commenter Edward says: “FWIW, I’ve done the grunt work with the Yahoo Pipes. You’ll need a Yahoo account, but once you have one you can simply Clone this pipe: http://pipes.yahoo.com/fieldofscience/full. With your Clone, go to Edit Source, then change the feeds in the Feed Fetch module to yours, and in the Simple Math module put the number of feeds you are combining.”
• XFruits. Seems to be very feature-rich. I don’t know of anyone who is using it.
• Dapper
• FeedWeaver
• FeedStitch
• FeedKiller. Looks very easy to use; doesn’t require a login; which means, I think, that the feed won’t be editable later; puts a feedkiller ad on each post.
• Feed Informer. Also looks really easy to use.

Software packages

If you have access to a web server and are able to set up a software package on it, you can run your own feed aggregator. Benefits: no ads inserted into the feed; you are in charge of the server and whether it is stable. Down side: you have to have some knowledge and a server.

• Planet Planet
• RSS-Aggregator
• FeedJack

Other tools

• Feedburner. Once you have a feed, you can use Feedburner to make a new URL for it. This can be nice a) because Feedburner provides usage statistics, and b) in case your new aggregated feed has an ugly URL.
• Feed Rinse. Filters feeds for you: “You can rinse your feeds by keyword, author, tag, etc, or filter profanity and more.”

Did I miss anything? I’m sure I did, but I’m happy to add more if you let me know what I left out.

How to create an aggregated feed

Hi. You don’t know me, but I’m here to try to help out with some of the technical aspects of science blog aggregation. I’m going to start by writing about how some bloggers might get together to set up a blog aggregation.

So: you are an independent blogger, and you want to aggregate your blog with some friends’ blogs, and then you want scienceblogging.org to aggregate that aggregation. What does that mean, and how do you go about it?

The people

The first step is to find a group of people who blog about similar topics at least some of the time.

You sometimes post about cognition, sometimes meta thoughts about science blogging, and sometimes personal ramblings, at ramblingscienceblogger.blogspot.com. Your friend Jane writes about neuroanatomy, addiction, and her young daughter at janesaddictionneuro.blogspot.com. Your friend Bob writes about behavior, neurotransmitters, and his dogs at bobsbehavior.wordpress.com. The three of you would like to create a “Brain and Behavior” aggregated feed. You agree that any posts in any of your three blogs tagged “neuroanatomy” or “behavior” should be included in the new aggregated feed. Posts that don’t have either of these tags won’t be included, although posts that have one or both of these tags and some other tags will be included.

For example, when Jane writes a post about the hippocampus, she tags it “hippocampus” and “neuroanatomy.” This post will be included in the aggregated feed. When she writes a post about her daughter, she tags it “kidblogging” only. It will not be included in the aggregated feed. Bob writes a post about a funny thing his dog did yesterday and tags it “ginger”; it is not included. Then Bob writes a post about how his dog’s behavior during thunderstorms reminds him of a recent article he read about fear conditioning in rats, and he might tag that “ginger” and “behavior.” That post would be included.

My examples assume that all the bloggers in this group are independent bloggers, but of course they could just as easily be bloggers on a network of some sort.

The aggregation

You choose an aggregator service to manage your new aggregated feed. You tell this aggregator service to aggregate the following feeds:

http://janesaddictionneuro.blogspot.com/feeds/posts/default/-/neuroanatomy

http://janesaddictionneuro.blogspot.com/feeds/posts/default/-/behavior

http://bobsbehavior.wordpress.com/tag/neuroanatomy/feed/

http://bobsbehavior.wordpress.com/tag/behavior/feed/

http://ramblingscienceblogger.blogspot.com/feeds/posts/default/-/neuroanatomy

http://ramblingscienceblogger.blogspot.com/feeds/posts/default/-/behavior

In other words, you are telling the aggregator service to pull in RSS feeds for Jane’s, Bob’s, and your blogs, but only the posts with the tags that you care about. You must include a separate URL for each tag and each feed — so for two blogs and two tags, you include four URLs. For three blogs and two tags, you include six URLs, and so on. The patterns demonstrated above will work for all blogspot and wordpress blogs.

The aggregator service then provides you with a new RSS feed which contains all the posts from Jane’s, Bob’s, and your blogs tagged “neuroanatomy” or “behavior.” You publicize that RSS feed however you want — you may just blog about it, or you may create a web page as a home site for it. You definitely let scienceblogging.org know to aggregate it (contact@scienceblogging.org).

If you later decide that you want to aggregate your meta science blogging thoughts with some other people who also like to write about science blogging in general, there’s nothing to stop you from having a second aggregated feed on an entirely different topic, with entirely different people, using your same blog. Just use different tags. In fact, one post could show up in both aggregated feeds, if it used the right tags.

What is a tag?

• Wikipedia entry on blog tags

Tagging is a way of noting the subject matter of a particular blog entry. Most blogging platforms will provide a way for you to tag each blog entry.

What tags should I use?

That’s for you and your co-bloggers to decide. It would be nice if bloggers started coming to a consensus on tag names for particular topics, so that aggregating different blogs by tag was easier. We’ll see if this happens.

What if my blog is not a Blogspot blog or a WordPress blog? How do I find out what the right format is for a feed for a particular tag?

Try doing a web search for “tag RSS” and the name of your blogging platform. Or comment here and I will try to help you figure it out.

What’s the point of this, anyway?

Subject aggregations are convenient for readers — a way to get an overview of blog posts by topic, rather than by author. They are also a way to build community, with several different authors working together to generate related content.

What aggregator services are out there and which are the best ones to use?

There seem to be several to choose from, but I don’t have experience with them to know which are better. If you have found particular ones that you like or don’t like, comment here and let people know. Yahoo Pipes seems to be widely used, but there are others.

Adding more blogs to Scienceblogging.org

This site works by collecting groups of science blogs. Since there are thousands of science blogs, there’s no way the site could function if we collected them one-by-one. But we think it’s important to have a way to add new groups.

Bloggers can and do form their own groups all the time. Some of them are temporary, like Blog Carnivals, and others are permanent, like ScienceBlogs, or Field of Science, or many others. We’re interested in both types! We already aggregate science blog carnivals, but we’re looking for more.

Adding more blog groups is a bit trickier. How do we decide which groups to include? We want to be comprehensive, but not so overwhelming that the site takes forever to load. We don’t want to waste precious front page space with groups that are abandoned or rarely updated. We want to make sure the groups we include are really collections of science blogs.

I’d like this post to be a place where we come up with a good way to decide how and whether to add new groups of science blogs. But even defining a science blog can be hard. I tried googling “defining a science blog” but came up with nothing. I guess it’s up to me to start. Feel free to offer corrections/amendments in the comments.

A Science Blog:

• Discusses science research, principles, philosophy, teaching, history, news, or other fields related to science
• Is not required to always discuss science
• Studiously avoids pseudoscience, anti-science, and denialism–except to critique them
• Strives for accuracy, and corrects mistakes when they are pointed out
• Does not plagiarize or engage in other unethical behavior
• Discloses any conflicts of interests, especially financial ones

How does that sound? Again, I encourage you to offer amendments in the comments, and I’ll update them here as need be.

Next, we need to establish criteria for admitting a new group of science blogs. Here are a few concerns:

• The group should consist only of science blogs
• With a few exceptions, a blog group should not include blogs that are already included in other groups on our site
• The blog group needs to have one RSS feed that aggregates posts from all its blogs in reverse-chronological order
• The RSS feed should link directly to blog posts, not to some intermediate site. The feed shouldn’t include anything that’s not a blog post — like comments, etc.
• The feed should be regularly updated. (We may need to come up with a general rule, like if it’s not updated at least 3 times a day, you probably want to form a bigger group before being included on the site)

Anything else? Again, let us know in the comments.

For the moment, Bora, Anton, and I will be making the actual decisions on which groups to admit, but we’re striving to be inclusive and comprehensive. Ultimately we may need a more formal way to decide whether or not to add a group. We may also need to come up with a system for removing groups that no longer work for this site.

Here’s where we open it up for comments. Feel free to suggest new groups, amendments to the criteria for including a group, and suggestions on how to administer this site in the long run. We look forward to hearing what you have to say!

[Update August 23: Removed authorship requirement and added conflict-of-interest statement]

Some thoughts about science blog aggregation

(Adapted from two posts on Word Munger)

After the summer’s “PepsiGate” affair and the subsequent departure of 20 or so bloggers from ScienceBlogs, I suggested that if the departing bloggers want to continue to have the kind of influence they used to have at ScienceBlogs, they need to do something more than just start or restart their old, independent blogs. They need to form a new network — perhaps built around different principles, but a network nonetheless. They might choose to have a central site based on RSS feeds or some other aggregation system, but there needs to be a systematic way to connect their conversations. Otherwise, most readers will tune out. It’s simply too much work for most readers to follow a diverse set of disconnected blogs. Social networking sites like Twitter can bring important individual posts to light, but are less effective at sharing the extended conversations that go on between blogs.

Sure, there are some other burgeoning science blog networks, but none seem to be prepared to assume the ScienceBlogs mantle (which ScienceBlogs itself hasn’t actually yet ceded). There are also some lists of all the bloggers who’ve left ScienceBlogs, but they don’t capture all the other science bloggers who were never a part of ScienceBlogs, or the many excellent bloggers who chose to stay.

To me, the obvious next step would be to find some way of collecting all these disparate voices in one place. Sure, ResearchBlogging does some of that, but it only captures posts specifically about peer-reviewed research, which is probably less than ten percent of what scientists and science communicators actually blog about.

One idea that shows promise, at least as a stopgap, is to use an existing social network to do the task. There’s already discussion over at Friendfeed about doing just that. The advantages of such a system is that Friendfeed already has tools in place to help people “like” and “dislike” posts, discuss them, and so on.

To see how this might work, I created a FriendFeed group for Anthropology, based on blogs registered with ResearchBlogging.org. You can check it out here. But this isn’t all Anthropology blogs, or even all Anthro blogs registered with ResearchBlogging — I cheated a bit because my default report of regisered blogs doesn’t include RSS addresses. I only used blogs from Blogger and Wordpress since their RSS URLs are easily reproduced based on the blog URL. And there are other problems. Many blogs cover multiple topics. How would you decide how which list(s) to put them on? What if someone started posting pseudoscience, or moved their blog? Who would be in charge of monitoring the list to make sure it remains useful? And how many people would actually register with FriendFeed just to follow blogs? The beauty of a site like ScienceBlogs is it stands on its own — you go there to read blogs about science. Someone who’s only interested in science (and not social networking) is less likely to hang around a site like FriendFeed just to read science blogs. I’m unconvinced that a set of feeds could have the same influence as a dedicated science blog aggregator.

In the wake of these thoughts, Bora, Anton and I came up with something we think is at least a little better. This site is sort of an aggregator of aggregators. We’re letting others do the work of collecting blogs into bundles; we’re just sharing all those bundles. If other bundles are promising, we’ll add them to the aggregator here, with a minimum of fuss. It’s not ideal — I think the ideal aggregator would have more active curators, and a way to sort through all the posts by topic — but it’s certainly a good start. Let us know what you think.