How are development organizations using Google Analytics?

In the previous post, I described the importance of having a digital analytics measurement plan and I presented some essential elements for correct and efficient use of Google Analytics (GA). However, recent work I’ve conducted make me wonder how advanced (or not) the use of digital analytics - and GA specifically - is amongst development organizations. My recent experience was limited to 6 organizations (different in size, resources and capacities) so the sample is clearly limited. But some trends are probably more common than not.

Google Analytics is not always used to its full potential 

In reviewing how different websites use GA, I discovered huge differences. For some organizations, the setup of Analytics is far from optimal. For example, one organization didn’t have a great understanding of the differences between GA account, profile and property, which resulted in unstructured proliferation of accounts.

The use of different reporting views, as well as filters and advanced segments is also not very common. This means that Analytics data are just analysed in aggregate, without telling you much about the specific audience you intend to reach. For example, if your website is targeting users in North Africa and the Middle East, you need to be able and single out traffic from these regions, to better analyze your target audience.

Tracking goals and ‘conversions’ is not always common practice. Goals can be set up in various ways in GA to track users’ interaction with the website - when they scroll on the page, click a link, decide to print a page, comment or spend a certain amount of time on the page. This can provide a great deal of information to website managers and editors, to improve the way information is presented and webpages are organized, as well to increase users’ engagement.

Only one out of 6 organizations stood out in using an advanced configuration of Google Analytics to create different reporting views, filter data, track goals and conversions.

Google Analytics too often stands alone 

I have highlighted before the importance of a digital analytics and measurement plan - and how Google Analytics may eventually just be a part (even if the most important) of your data collection and analysis system. On the contrary, I didn’t find a lot of this in the websites I’ve recently reviewed.

On the one hand, while Analytics is the default tool to track digital analytics, in most cases is also the only monitoring tool. On the other hand, when digital analytics are collected from different sources, (e.g. website, newsletter, RSS feeds, social media, etc), more often than not they are not presented and analysed in aggregate. Finally, not all organizations are regularly producing actionable reports on the basis of their analytics, to inform future actions and improvements on the website.

Only one organization presented a more advanced understanding of its’ digital analytics process, with multiple data collection points (e.g. website, newsletter, RSS feeds, social media, etc) that fed into a dashboard spreadsheet, using formulas and calculations to avoid double counting and over-reporting of metrics. Even if there was no document describing a strategy, this is already a great step towards more efficient use of digital analytics.

What can be done? 

I think a lot could be achieved through the availability of more specific content for international development and the open exchange of experiences around digital analytics for the development sector.

The majority of information and guidance available online, while comprehensive, in general tends to focus on e-commerce and more business oriented websites. Other sources such as the Digital Analytics Programme (DAP) provides a good example of guidance and best practices, training and support in digital analytics. However, the target audience is also very specific, DAP being designed for US Government government agencies that provide information to the public. Eventually, there is not much available that focuses specifically on digital analytics for development - and information and knowledge services specifically.

Secondly, I think website administrators and managers should be more open about how they do digital analytics, as ODI has been doing by sharing their M&E dashboard. Knowledge sharing and learning opportunities should be created for users to exchange notes and learn from each other, to identify good practices and examples that can be replicated. Ideally, I believe that web managers should also be open about the actual number of their website stats. Especially for publicly-funded websites, this would mean more transparency and the possibility to compare and benchmark different websites.

Finally, I think donors should play their part in fostering better use of digital analytics in projects and programs they fund. Besides acting as convenor for peer learning initiatives around good use of digital analytics, donors should provide stronger guidance and support in this area, to make uniform data tracking and collection across different projects. Ideally, for donors’ funded websites and knowledge services, there should not just be the mention of few, poorly selected web metrics in the project logframe. A digital analytics and measurement plan should be developed as part of the project inception phase.

In the next post in this series, I’ll look specifically at what metrics and indicators could be most useful, amongst the dozens available, for development websites and knowledge services.

Understanding digital analytics

In the past few months, two different projects gave me the opportunity to spend quite some time working on Google Analytics. The projects were different in scope: in the first, I was asked to review the use of Google Analytics as part of the monitoring and learning system of a think tank; the second project was part of a larger evaluation of development information services, to understand the reach and use of different websites. Overall, I was able to review how 6 different development organizations are currently making use of Google Analytics to track web traffic and users’ interaction.

This is the first post in a three post series where I’ll share some of the learning from this recent work. I’ll start with some key points for efficient use of Google Analytics (GA). Then in the second post I’ll present examples of how different organizations are using GA. Finally, in the third and final post in the series I’ll zoom in more into Google Analytics metrics - and measuring what matters to you.

Google Analytics - from basic to advanced use 

GA has developed a rich and sophisticated toolset over the years. It is now one of the most commonly used tools to monitor website traffic and engagement. It’s probably the industry standard for web analytics across different business domains, such as e-commerce, government, education, and development, too. While some have questioned the accuracy of Google Analytics and there’s no shortage of alternatives tools out there, in my opinion Google Analytics remains one of the most powerful tools (for everyday use, especially in smaller organisations). One of the aspects I like most about Analytics (besides the fact that it’s free…) is its continuous improvement - in terms of its own features and functionality as well as integration with third party tools (such as Supermetrics), and integration with Google Sheets add-ons.

Help! 

I recognize GA features can be overwhelming: with all the information GA can track it’s easy to drown in a sea of data. Luckily, a simple search in Google will return a lot of results pointing you to tutorials, guides and training videos that will allow you to go well beyond a basic knowledge of Analytics. If you’re new to Google Analytics, or want to take it to the next level, I suggest you take a look at these resources:

• Google Analytics Academy - with dozens of video lessons covering from basic Analytics to mobile Apps Analytics. 
• Google Analytics checklist for new projects - a quick but comprehensive guide on how to set up Analytics to track a new website. 

As a minimum requirement, your correct installation of Analytics should include:

• A clear and well organized set of Analytics properties, profiles and views. 
• Custom filters and segments that you can apply to split data and focus on your key audiences. 
• A set of website goals and events that will allow to understand better actions and behaviours of your site visitors. 

Analytics measurement plan

It’s relatively straightforward to get your Analytics set up properly and tracking data. But this is just a small part of the job. In fact, even before you get going with Google Analytics (or any other web analytics software), what you need is a measurement plan.

A measurement plan is a document that:

• Defines your business objectives and outcomes you want to see; 
• Presents the strategies and tactics to reach these outcomes;
• Illustrates the metrics you need to monitor and the tools and processes to collect data; and
• Includes goals and targets for your selected measures. 

In this sense, Google Analytics is only one of the different tools that you have to measure your objectives and outcomes. If you’re active on Facebook and Twitter or publish video on YouTube, these should also find a place in your measurement plan.

There’s a lot of good resources out there on digital analytics and measurement planning. You can check Google’s own guides or this post from analytics specialist Avinash Kaushik. Finally this guide presents a great intro for non-techie on how to create a measurement plan for Google Analytics.

So as essential as a measurement plan may seem, my recent work experience tells me that this is probably more the exception than the norm in development organizations. I’ll discuss more about this in my next blog post. In the meantime, how would you consider your current use of Analytics? Are you using it to its full potential? Do you have a digital measurement plan? Let me know in the comments below!

Four useful tools to monitor Twitter use and engagement

If you search on Google for “Twitter monitoring tools” you’ll probably get a lot of blog posts and resources with links to tools such SocialBro, Crowdbooster, TwitterCounter and many more. And of course don’t forget Twitter Analytics itself, available to all users for a little over one year now, that allows you to track engagement on your timeline activity and followers growth.

However, for most popular uses there are limitation in terms of free use and you’ll have to purchase a subscription to use all their features. And Twitter Analytics doesn’t give you all you may need in terms of tracking Twitter use and engagement. For example, you may need to track activity around a specific hashtag or Twitter account, around an event or an online campaign. Or to analyze the Twitter users in a specific list. Or to monitor the number of followers and growth of several accounts, some of which are created and managed by others.

For several of the projects I’ve been working on in recent months, I had to spend some time looking at possible solutions to these needs. After some researching, testing and experimenting, here are four useful tools that you can use to enrich your digital analytics around Twitter.

Keeping track of Twitter followers numbers 

Example dynamic followers count 

 While number of followers to a Twitter account may not be by itself very meaningful as a metric to track users’ engagement, it is still a quick, useful indicator of your ability to connect and expand your audience. Tracking Twitter followers numbers is easy when you have to look at the followers of just one Twitter account. But what if you need to monitor and aggregate the followers of several Twitter accounts, how can you easily do it?

After some searching and (several) failed tests with different solutions, I finally came across this excellent Google Script created by Sarah Marshall. After you setup and configure the Script, you’ll be able to automatically add Twitter followers numbers to a Google Spreadsheet, for as many Twitter account as you need.

This is a great tool to have a quick overview of the followers of different Twitter account at a glance. Moreover, Sarah has kept updating the Script, whenever Twitter has changed the way its API works.

Exporting Twitter lists 

TwExList is an excellent service to export a wide range of information from Twitter. For example, you can export information such as followers and following, tweets, mentions and favourites of your own Twitter account or of another user account. Likewise, you can export information around a Twitter list, regardless of whether you have created the list or not.

Example TwExList Excel export - click to enlarge the image

Very simple to use, in four easy steps TwExList offers you different options - from Excel and CSV to html - to export the data you’re interested.

Given the many different types of information that you can export using TwExList, there are many, many ways in which this tool can come in handy. For example, you can analyze your Twitter timeline posts, to see which one created more engagement (retweets and favorites); or you can inspect your followers - or the members of a list - to see influencers and active users . Finally, you can use this service to keep an eye on your ‘competitors’ and see how they are using Twitter, the kind of following they have and how they create and sustain engagement.

The service is free to test, limiting the export to the last 50 tweets in a list. Different pricing options are available from single to unlimited exports.

Monitoring Twitter engagement 

Two very useful tools you can use to measure engagement and track conversations on Twitter are Zapier and TAGS.

Zapier connects web services and apps to automate ‘actions’ between them on the basis of defined triggers. For example you can use Zapier to connect Twitter and Google Drive so that each new engagement action (mention, RT or reply) around a Twitter handle or around a hashtag is recorded and added as a new row in the connected spreadsheet. Amongst others, information that can be recorded includes: post link; user name and location, RT counts, etc...Pricing plans for Zapier start with a free account, for 3 Zaps and up to 100 records/month. Paid accounts start at $15/month for 10 Zaps and 3000 records/month.

Use Zapier to connect Twitter and Google Sheets - click to enlarge

A great alternative is TAGS, the free service developed by Martin Hawksey. TAGS (short for Twitter Archive Google Spreadsheet) is a Spreadsheet Template that users can easily copy and configure to suit their needs to track mentions and search terms on Twitter, and automate the collection of these results.

One of the things I love about this service is that, together with a worksheet collecting the various tweets, the spreadsheet template comes with a summary and dashboard sheet already built in, so you don’t have to worry about setting this up but can start focusing on the analysis of your results immediately.

Example of TAGS dashboard

Additionally, TAGS comes with two nice add-ons: a searchable, online archive of the tweets collected and an explorer to visualize the engagement and connections between different users.

Example of TAGS searchable archive - click for live version

While I’ve started using TAGS as part of the toolkit to monitor online engagement for social reporting around events, increasingly I’m recommending it as a good solution for ongoing monitoring and evaluation of a specific Twitter handle, or a particular hashtag or search term.

What other tools do you use to support and complement your Twitter Analytics?

Measuring the quality and reach of a curated news highlights service

In my previous blog post, I introduced the process and tools we’re using to publish a monthly curated news update about about water, sanitation and hygiene (WASH) as part of the work we are doing to support KM in the Building Demand for Sanitation (BDS) programme.

With 9 monthly issues published so far, how do we measure if this activity is working well, and grantees are using the curated news put together by their peers?

As we’re using several tools to run our Curated Updates, we also make use of several metrics and indicators that form our basic M&E system. But before we dive into this - and what we’ve learned from our own monitoring and evaluation - I think it’s important to mention that this activity is intended for a rather small audience, with an initial list of around 50 recipients and website that is purposely not indexed in search engines.

Nevertheless, some useful lessons can be identified also from a small project like this one, and maybe scaled up to larger initiatives.

How useful is the service? 

From what we can see, users make use of this curated news update. They do open the newsletter that hit their mailboxes regularly during the first week of each month. Out of the many different things you can monitor in MailChimp, we decided to focus primarily on open rate and click rate. As you can see from the image below, MailChimp provides you also with benchmarks and average for your industry, and we’ve been pretty happy to have an email open rate which has constantly been above what MailChimp consider the industry average.

Besides the generally positive informal and anecdotal feedback that we receive from users, we can also see that the they appreciate the service: the list of subscribers keeps growing, in spite of the fact that we don’t do any active promotion of this service. So the list growth is all driven by peer recommendations and suggestions, with subscribers encouraging their peers to register for the service as they find useful to receive a monthly newsletter with a few, key resources that are already filtered and digested by experts, and whose judgement they can trust.

Are users clicking on the suggested links? 

In general, resources shared through the newsletter are opened and users go and read the original document. MailChimp provides automatic tracking of the links included in any newsletter. However, for some technical reason, this didn’t work at first with the RSS driven campaign, and we failed to properly track clicks on links in the first few issues of this Curated Updates service. So while figuring out the issue (which we did eventually, thanks to the great MailChimp support team) we started using Bitly to shorten the links for each curated update resource included in the newsletter.

This gives a better idea of what resources resonate with our audience, where the click happens (if on the newsletter, on the website or elsewhere) and where the audience is located. Through Bitly, we can also see if the link is re-shared on other social media channels, allowing us to have better link tracking and potential insights into where our audience interact online.

How valuable are the resources? 

The RatingWidget Wordpress plugin allows us to get a better feeling of the usefulness and relevance of each single curated news item for the users, and to understand the appetite they have for different types of contents. As we’re using the pro-version of this plugin, the ratings are nicely collected and displayed in the RatingWidget dashboard, as illustrated in the image below.

As you can see, the number of votes remains low and we find it challenging to encourage users to make more use of this feature. However, we believe it remains a valuable addition to our M&E for this activity. Ideally, the perfect scenario would be to include the rating widget directly into the newsletter campaign, but this doesn’t seem possible from a technical point of view - or at least, we haven’t yet figured out how to do it!

Publish, review, adapt, publish...

This activity has been planned around a “publish, review, adapt, publish” cycle for 12 months, and indeed for each of the issues we’ve been publishing so far we’ve been tweaking and adapting the newsletter, the way we present the content in it and online, as well as some of the tools we adopt to track users’ behaviours and preferences. Each time, we try to learn something and put it into use in the next issue - the curators from a content point of view, looking for resources that resonates more with the audience; and us from a technical point of view.

We’re running the service for a year, which comes to an end in July 2015. As we review how it worked we’ll share what we learn and update our findings on this blog..

Supermetrics: how to easily collect Analytics and YouTube data

Collecting and charting data from your YouTube and Google Analytics can be a time consuming process and not a very enjoyable one, as I’m sure you are aware of...But there’re some good news here. And it’s called Supermetrics.

What is Supermetrics? 

Supermetrics is a Google Drive add-on - an extension that you can add to your Google Sheets and Docs. Add-ons are similar to Google Apps Scripts but with some differences. If you’re curious to know more about this, go over this blog post here, where the two are compared in detail. And where you can also find a good overview of some useful Add-ons.

With Supermetrics you can create queries to (semi)automate your reporting from sources such as Google Analytics, Youtube, Twitter, Facebook, Google AdWords, etc….Data from these sources are added to a Spreadsheet (or Google Doc). You can then create live charts from these data, so they are automatically updated once you refresh your Supermetrics queries. Supermetrics comes with a free and pro versions.

The main two differences are about the amount of data you can get from each query and whether you can schedule automatic refresh of your queries, which is available only for the pro version (49 USD/month).

In CARIAA program M&E dashboard, we used the free version of Supermetrics.

How to activate and use Supermetrics?

To use this tool follow these simple steps below here:

1. Activate Supermetrics - Navigate to the Add-ons button in your Google Sheets and browse the gallery to select Supermetrics.
2. Set up your queries by 
1. Connecting your data source and selecting your profiles - Launch the Supermetrics sidebar and connect the various sources you want to monitor. In our case, we only connected the Google Analytics and YouTube profiles that were shared with the Google Account we used.
2. Selecting the date range you want to monitor and the metrics you are interested in - In our case, we set up queries for a specific time frame but you can also have a year-to-date query, last month, etc...Out of the many Google Analytics and YouTube metrics you can collect, we limited the query to web sessions, users and pageviews for Analytics, and views for YouTube.
3. Launch your queries and see your data - Once you’re all set, decide how you want to get your data, in a table or in the various default charts. Data will be downloaded and added to your spreadsheet.
4. Refresh your queries - From the Supermetrics menu under your Add-ons tab, you can refresh all your queries with a click and update your results and charts.



Launch Supermetrics from your Add-ons menu

5. Modify and edit your queries - Alternatively, you can click on Manage queries; this will open a worksheet where all your queries are listed. From here you can manage and modify them, for example changing the date range of the query.

Using Supermetrics has proved to be a very effective and efficient way to collect usage stats and traffic data for multiple Analytics profiles and YouTube channels at the same time, without having to waste time navigating in and out these various accounts to get individual data.

Note that, while at the beginning of this post I mentioned that this tool can also be used to collect data from Twitter, in our case we decided not to use Supermetrics for Twitter.

In the next post, I’ll explain you why we took this decision, and what we used instead. In the meantime, if you haven’t try it yet, give Supermetric a shot and see for yourself how it works.

Power up your Google Sheets with Apps Scripts

Google Sheets and Docs are very powerful, flexible tools for data collection and analysis. But do you know that there's a lot more you can do with both Sheets and Docs, using free tools or just a bit of extra coding? And even if you are not a programmer? Do you know you can:

• Enable users to edit responses they have made in Google forms? 
• Automatically copy (part of) data from one Sheet into another one? 
• Simultaneously collect various metrics for your Google Analytics, YouTube and Twitter accounts?
• Automatically track Twitter posts around a Twitter handle, hashtag or search term?
• Automatically count the number of Twitter followers of various accounts and add them dynamically into a Google Sheet? 

In this post and the next ones, I'm presenting few different options I’ve used to 'extend' Google Sheets and how I used them in the development of a program M&E system and dashboard for IDRC.

Today I'll look specifically at two possible use of Google Apps Scripts for Google Sheets.

Google Apps Script editor

About Google Apps Scripts 

Google Apps Scripts "is a JavaScript cloud scripting language that provides easy ways to automate tasks across Google products and third party services and build web applications."

With Apps Script, there's quite a lot that you can do, such as write custom functions, create macros and menus for Google Sheets. Google itself provides quite some guidance on how to work with Apps Script, but sure this may not be easy for a total beginner.

Luckily, there's plenty of kind (and clever!) people out there that have developed Apps Scripts and make them available to others online. And those you can just use!

Use Apps Scripts to collect Forms "edit response links" 

In M&E system and dashboard developed for IDRC program, as we saw part of data collection is manual, with users inputting data for research outputs and pilots through a series of Google Forms. So what if users want to update/modify an existing entry?

If you are familiar with Google Forms, you probably know that responses can be collected into a Sheet. You may also know that you can set up your Form so that, after an entry is submitted, it sends an email to the person that has contributed that submission. The email contains a link that the person can click to in order to modify/edit the entry.

Well this is sure nice and useful!

But wouldn't it be better if the edit response links were also added to the Sheet where the responses are collected, nicely ordered in line with the relevant form entry?

You can do this, with this Google Apps Script I've found browsing online.

To use this Apps Script, what you have to do is the following:

• Click on Tools >> Script Editor in your destination Sheet (as in the image on the right);
• In the Script Editor, copy this piece of code here; 
• Change the parameters as indicated in the code; 
• Save the script and run it; 
• Click on Resources >> Current projects trigger and set the script to trigger at every new Form entry; 
• Check that the edit response links are added in the right column on the destination Sheet.

Done! You set it up once and the script will continue to run and collect edit response links every time new responses are added via the form.

Importing data from a different spreadsheet using scripts

While this Apps Script is very specific for when you use Forms, there are few others that can come in handy in more occasion. For example, to automatically copy (part of) data from one Sheet into another one.

While you can actually do also this using in-line cells functions, as nicely explained in this post, I've found this to be not very reliable and not to always update automatically. So I would recommend to take the slightly more technical route and use Apps Scripts. You can find the link to the code and the explanation on how to insert this script.

Give it a try and see for yourself how it works. And let me know in the comments here if you are using other useful Apps Scripts that’s worth sharing.

How to create an M&E dashboard using Google Apps

Last year we did a fair amount of work with IDRC to set up a KM platform for a new collaborative research program. In the follow up to that project, we developed an M&E system for the program, using the same technology infrastructure used to build the platform itself - the Google Apps for Business.

After last week’s case study on building the R4D dashboard with Tableau public, in this post I’m presenting how to set up a M&E system and dashboard using a combination of various Google Apps and free third party tools. This post is very much an overview of the process and the final product we delivered. In the next blog posts in this series I’ll look at the specific tools used from a more technical perspective.

M&E Dashboard - Click to enlarge

Who needs a dashboard, and why? 

This IDRC program is made of 4 different research consortia and the IDRC program team in Canada. Further, each consortium works on a specific issue related to climate change and adaptation. In doing so, it brings together different organizations geographically dispersed. So as collaboration is the basis of the program the M&E system had to follow this principle. So our brief was to “design platform-based, collaborative tools to collect monitoring data on up to eight key indicators in the Monitoring Framework.”

Ultimately, these data had to be brought together into a M&E dashboard that could be easily shared with donors and program leads as a link or quickly printed in PDF at regular intervals. As for the R4D dashboard, also this dashboard had to provide a “snapshot of progress against key indicators in the program's monitoring framework using data entered by consortia and the IDRC Team.”

So what are these indicators?

What to measure? Theory of change and monitoring framework 

The program M&E working group had already produced a solid Theory of Change with three clear objectives; for each they had defined the dimensions and potential metrics to be included in the M&E system. This made our job easier as it was clear from the outset what had to be measured and for what purposes. So we just had to help the team unpack a bit the various metrics and dimensions, and define the exact indicators and values to be tracked in the system:

• Research outputs and pilots, including indication of type of outputs, authorship (gender and country), quality of outputs and their accessibility (whether peer-reviewed and/or openly accessible on the web), etc... 
• Web traffic, social media and engagement data, such as web sessions and downloads, Twitter followers and number of conversations and Tweeps around specific accounts and Hashtags, media tracking, number of events and participants rating, etc. 
• Grants and awards distributed, including gender and location of recipients.

M&E Dashboard - Click to enlarge

When and where? Data collection process and storage 

While the system (and resulting dashboard) was planned to be updated quarterly, we agreed on the principle that data collection would be automated when possible, and manual when other solutions were not at hand. As a result:

• Data around web traffic and social media are automated or semi automated, using a series of third party tools and applications (I’ll talk about this specifically in the next blog post) 
• Data around research outputs, pilots, grants and awards are entered via users’ submission forms, using Google Forms. While forms can (potentially) be submitted by anyone who has a user account on the KM platform, in reality specific users for each consortium are responsible for this process, while others are responsible for quality control, to ensure that entries are complete and there are no duplicates. 

Regardless of how the data are collected - manually, automatically or semi-automatically - they all feed into one of the 3 separate log files set up for the the three objectives in the Theory of Change. Google Spreadsheet are used for these log files, and the appropriate sharing and editing permissions are in place.

How to display the data? Platform, design, prototype and production 

On the basis of an initial sketch of the dashboard produced by the IDRC team, we populated the log files with dummy data and produced two different prototypes, one using Tableau Public and one using Google Charts and publishing them into a Google Site. We agreed to use Google tools to avoid adding another layer of complexity to the system and keep it all inside Google Apps. Additionally, as Charts are generated by the log files, when the log files are updated so are the charts on the live site, which is a great short-cut, cutting down work on updating the dashboard.

Similarly to the R4D dashboard, this program dashboard presents a tabbed navigation at the top, with one tab for each of the objectives monitored in the framework. This way we could present objective-specific charts, tables and figures in a clean, uncluttered interface.

Additionally, the main tab of the dashboard presents what we called ‘curated content’, such as a selection of recent publications, blog posts or key events that are hand picked by the dashboard administrators to highlight specific information.

M&E Dashboard - Click to enlarge

What next? Possible platform iteration and next blog posts 

This dashboard became at the beginning of 2015 and its second update is planned for the end of this quarter, so it’s too soon to evaluate it and think about possible iterations. However, feedback received from users has been positive so far and the system delivers the required information to the different target users.

In my opinion, a possible way to improve it would be to add filters and controls to the charts currently published on the dashboard, so that users can interact with them, browse for specific period of time, make comparisons, and get more out of this visual representation of data.

To do this requires working with Google Apps Scripts, JavaScript cloud scripting language that provides easy ways to automate tasks across Google products and third party services. I’m not a programmer but I like learning new things and findings solutions that others have already implemented. So also in the current version of this dashboard I’ve made use of Google Apps Scripts to collect data and to copy them from one spreadsheet into another.

If you are interested in what Apps Scripts I’ve been using and what they can do for you, subscribe to the blog and sit back till you’ll get my next post in this series - or share your experience in the comments below here.

R4D dashboard: Visualize web access to DFID funded research

Collecting traffic and usage statistics for a website or portal can be a very time consuming and tedious task. And in most cases you end up compiling monthly or quarterly report for managers and donors that will be shared as email attachments - and at best skimmed, since there is so much information. But there are smarter ways you can do this process and bring life into your data, as as I explained in my previous blog.

Our case study is the R4D portal, a free access on-line portal containing the latest information about research funded by DFID, including details of current and past research in over 40,000 project and document records. Until 2013 we were part of the team supporting and managing the site.

As part of our work packages, we developed an online, interactive visualization of web traffic and usage of the R4D portal and its social media channels. The R4D dashboard, built using Tableau Public, is still updated and in use. However, since the termination of our support contract, it hasn't been iterated and improved since 2014.

This posts presents the process we followed to develop the dashboard, the tools used and the lessons learned in what was very much a learning by doing journey.

R4D Usage Dashboard 

 Why develop the R4D dashboard? 

The collection of usage and traffic data for R4D used to be pretty much standard: a series of excel files updated monthly to generate charts and graphs. They were then put together in a PDF report and shared with project leads at DFID. The idea to develop instead an online, public dashboard of R4D web traffic and usage was inspired by the excellent work from Nick Scott and ODI, which he shared with us during a Peer Exchange session we organized back in 2012.

Donor organisations such as DFID collect a lot of statistics and indicators but these are often kept within projects and programmes and not made available for all staff, as was the case for R4D. So the reason behind the R4D dashboard was primarily to open up our stats and make them more accessible to anybody interested in it, not just the people that had sign off on the project.

Also, by encouraging a more open approach to web stats, the idea was also to have more terms of comparison: it is difficult to evaluate how well your website is doing if you can only compare against yourself. So being able to see how much traffic similar websites are generating will help you assess your own effort and performance.

So what did we do?

Process wise, we pretty much followed the steps indicated in my previous blog posts. With the primary audience well in mind, we started to select the metrics to include in the dashboard:

• Website stats: Visits and visitors; referring sites; visitors by country; PDF downloads and top pages. 
• RSS feeds subscribers Twitter clickthroughs and Facebook Insights data (later removed)
• Number of outputs added to the R4D database (by type, for example open access articles, peer review articles, etc…) 

We decided that it was feasible to collect this data monthly as xls or cvs files exported to from the site(s) and save them into a shared Dropbox folder. This was the most effective way as data collection was decentralized with different people working on different platforms. With our limited budget, it was not possible to automate the data collection process, so this was entirely manual.

Software platform selection took quite some time in the initial phase of the process. We selected Tableau Public as our dashboard platform, and then had to invest more time in learning its features and functionality. But it was totally worth it!

Why Tableau? 

Tableau Public is free software that can allow anyone to connect to a spreadsheet or file and create interactive data visualizations for the web. There are many tutorials out there if you just Google for it, so I’m not going to tell you here how it works in details. But here are my top reason for using Tableau Public:

• It's free! Well, that's a good reason already if you don't have resources to invest in business intelligence or visualization software - and normally the cost for these are steep and way outside the budget of the organizations we work with; 
• It's intuitive. You don't need to be an expert to use the tool. The interface is very simple (drag and drop) and you can easily find your way around. 
• It's rich and deep. There are so many charts you can choose from and you can play around with different visualization until you are happy with the result. It also goes much deeper than Excel with analysis and interactions.

What did we learn? 

Besides learning how to use Tableau Public itself, here are the main things I learned along and around the process of developing the R4D dashboard:

• Google Analytics is the industry market standard - but it tends to under-count your traffic.
  We ran two different website analytics packages on the main R4D portal - Google Analytics (GA) and SmarterStats - and noticed a huge difference in the results, with GA massively under-counting visits and visitors. So it's always worth installing another tracker to be on the safe side. 
• Updating Tableau is quick - but getting the data manually isn't
  Once your dashboard is set up, the process of updating it with new data is rather quick, just a few clicks and you are done. However data collection from the various sources in our case was mostly manual and it can be time consuming (and not much fun either!). If I were still working on the project, I’d look into ways to automate as much as possible data collection - while also looking at what additional (useful) data I could collect in an automated way. 
• Build it once - and then you *must* iterate
  When you're done building your dashboard, you're actually not done. We had a couple of iterations before arriving at the product that is now online. And I'm sure this would be different now had the project continued. This is because you have to evaluate whether the visualizations in the dashboard are actually useful and provide you actionable insights that can inform your strategy. Or simply because the software keeps evolving and can give you new possibilities that were not there before.

In the next post on this series I'll present a different approach to develop an M&E dashboard, this time using a combination of Google Forms, Sheets and Charts, together with Google Apps Scripts and Google Docs Add Ons.

In the meantime, if you have experience with Tableau or use other tools to create interactive dashboards, why not share it in the comments here?

How to create a monitoring and evaluation dashboard

So you have a website, a blog, the usual social media channels on Twitter/Facebook/YouTube, maybe a series of RSS feeds. On top of this, your organization or research programme also publishes original content, or indexes content produced by others into an online portal. And you also organize events and workshops and maybe offer grants and awards.

With all these online spaces, outputs and products that you produce, how are you going to collect and aggregate this data as part of your monitoring and evaluation activities? And how are you going to display and present it in an effective way that can be easily understood by your co-workers, managers and donors?

For the past couple of years, I’ve been experimenting with tools to display data and information in online dashboards. This post presents a short introduction to the topic. It’s the first of a series of posts that will look into online tools for data collection, storage and display.

What is a dashboard? 

According to Stephen Few’s definition “A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.”

More generally, a dashboard is a visualization of data related to the operation and performance of an organization, program or service. Dashboards are helpful tools to provide you with a quick overview to see whether you’re on track to reach the objectives stated in your logframe or Theory of Change.

Note that information about a wide range of channels can crowd one screen. So it’s important to be flexible and keep users in mind - keeping scrolling *very* limited and using features like tabbed navigation to view different set of metrics and indicators.

What are the steps to follow to build a dashboard? 

The Idealware report Data at a Foundation's Fingertips: Creating and Building Dashboards presents an excellent and detailed step-by-step description of the process to design effective dashboards for non profit. Ultimately the process boils down to 4 main phases:

1. Define your audience
   Of course this is absolutely critical, determining the way you design it, the graphs you include, their order and sequence. The dashboards I’ve developed in the past were mainly designed for managers and executives, to tell them about the progress of a program or service at a quick glance.
   
2. Identify the metrics to display - and how you collect them
   With the tons of metrics that you could collect, and the space limitations of a dashboard it is important to agree upfront which ones will be displayed in the dashboard. So it requires a bit of negotiation to agree upon what’s in and what’s out. Of course, the metrics should be useful in terms of monitoring progress towards the objectives in your logframe and theory of change. In this phase it is also important to discuss collection methods, frequency and access. 
• Are there any process that you can automate? 
• What is only possible instead through manual data collection?
• And is it realistic to collect this data monthly - if the properties are high traffic or include active campaigns, for example, - or is quarterly more realistic?
• Where are you going to store the raw data and who should have access to it?
  
3. Identify your dashboard platform
   This is a maturing market so there are a lot of possible solutions - from expensive business intelligence software to low cost or free tools. Generally the decision is defined by the resources available as well as the time you and your users have to invest in learning new tools. Note that while potentially you can build a dashboard in Excel, investing some time in learning how to use a powerful and flexible dashboarding tool such as Tableau Public can enable you to design more complete and effective dashboards.
   
4. Sketch, prototype and roll out
   In the design of the dashboard you need to find a good balance between the amount of information you want to display and the limited space available. So you have to carefully decide what graphs and chart you will use, what explanatory text you should include, which colours to use when...This will take a lot of testing and iterating to find the optimal design. Bottom line, your final product should: 
• Be simple, intuitive, easy to read and understand; 
• Present data together from different sources in an uncluttered way and following a logical sequence or order; 
• Offer a quick overview of the key metrics and indicators to assess progress towards the objectives of your program/organization/service. 

In the following posts, I’ll be presenting two case studies about work we did recently on visualizing monitoring and evaluation data into online, interactive dashboards. I will look specifically at the tools  used to put these dashboards together, as well as the individual tools used to collect and store individual indicators and metrics. 

For the more techie readers, I’ll also share the details of what I’ve learned recently using Google Apps script to automate some data collection and storage processes, as well as tips and tricks to monitor activities and engagement around Twitter, which I’ve been experimenting a lot with lately. So stay tuned!

How ODI uses digital tools for measuring success in research uptake

A fascinating peer exchange last week in DFID discussed issues (and possible solutions) to the challenges related to monitoring and evaluating research uptake and communication efforts. The session, facilitated by my colleague Pete Cranston, was the first in a series of 3 meetings organised as part of the R4D project, to address different aspects of social media and engagement in and around development research.

The conversation was started by an excellent presentation from ODI’s Nick Scott. Over the past months, Nick has put a lot of effort into looking at how ODI was doing its own M&E of research uptake. As a result, the organisation has adopted a new analytical framework - and a new set of tools to track usage and uptake of their research outputs. The ODI approach and dashboard is well described by Nick in a recent post and presented in the slides below - if you have missed them and are interested in the subject, make sure you have a read both of them.

ODI's M&E approach and dashboard on Prezi

What the ODI’s experience shows is why and how research organisations should escape the “tyranny of downloads”. Downloads and pageviews alone are simply not enough to capture the different ways in which users interacts with the digital content in today’s social web. For example, for ODI a good 10% of their traffic is generated by social media. Tweets, shares, Facebook likes they all add a new layer of information that needs to be taken into account. An efficient use of digital tools can help in capturing much more of this information and do a better job in assessing the uptake and impact of research outputs.

Additionally, even when you are able to track all these elements, you need to make up your mind and decide on which metrics you are actually going to focus. When it comes to Google Analytics for example, ODI focuses on just a couple of the many different things you can track - unique visitors, entry pages, country of origin and a few others. But the choice of metrics and indicators is very much related to the goals you want to achieve, and how the information gathered can be put to use.

But why track stats and pile up information if you can’t make use of it? ODI M&E reports are now more intuitive to read, more colorful and graphic and able to present insights in easy formats. They are shared throughout the organisations in a much more open way. Further, because the different statistics are tracked at the level of the single outputs produced, they provide ODI’s communications team with the evidence to back up what its already known, but not always proved: using a balanced mix of channels and a more comprehensive communication approach creates a positive circle that gives more mileage to the research outputs. ODI researchers are now more inclined to write opinion pieces and blog posts that complement research reports, or to tweet about their new publications.

Finally, looking at the specific technical implementation that ODI has put in place, this seems to be rather advanced and complex. It uses APIs, servers logs and business intelligent applications to combine a set of different statistics into a structured dataset that can be enquired at any given moment though a dashboard interface. This is the result of 6 months of work and quite some investment in time and technology, so it wouldn’t probably be a viable solution smaller organisations. However, as Nick suggests in the video below, there is a huge need of a ‘think tank M&E’ tool that is affordable and allows to track the footprint of research outputs across the social web. Hopefully we won’t have to wait long before such an application is available.

Social reporting at the KM Impact Challenge

The KM Impact Challenge is a collaborative learning process that has brought together practitioners to describe and share their experiences in assessing their knowledge and learning initiatives. Over 70 practitioners from different fields of expertise attended the unConference that took place in Washington, DC, on May 5 and 6, 2011, to discuss these experiences and debate tools and approaches to measure the impact of KM activities.

Euforic Services worked with GDNet to provide coordination and support for the social reporting during the face-to-face event.

We started with a core team of 3 to 4 social reporters, but soon after the event started, many more joined in the effort by sharing their impressions, particularly via Twitter, and contributing to the digital footprint of the event #tag #kmimpact.

During the two days, we recorded some 30 video interviews with speakers and participants, capturing their stories, impressions and what they had been learning at the event. We also posted several updates on the KMIC blog, featuring some of the stories emerging from the different sessions and embedding videos and presentations shared. Lastly, we produced a daily summary of the highlights from the unConference, aggregating and curating the best of the social content produced by us and others around the event and publishing it online.

You can see more about the event on the KM Impact Challenge blog and read this post from Sophie Alvarez from CIAT where she reflects on the event and share her main takeaways.

You can stay connected with the KM Impact Challenge by joining the online community, conversing on Twitter (#KMImpact) and Facebook, and subscribing to the RSS Feed and email alert.