Tuesday, August 30, 2016

How are development organizations using Google Analytics?

In the previous post, I described the importance of having a digital analytics measurement plan and I presented some essential elements for correct and efficient use of Google Analytics (GA). However, recent work I’ve conducted make me wonder how advanced (or not) the use of digital analytics - and GA specifically - is amongst development organizations. My recent experience was limited to 6 organizations (different in size, resources and capacities) so the sample is clearly limited. But some trends are probably more common than not.

Google Analytics

Google Analytics is not always used to its full potential 

In reviewing how different websites use GA, I discovered huge differences. For some organizations, the setup of Analytics is far from optimal. For example, one organization didn’t have a great understanding of the differences between GA account, profile and property, which resulted in unstructured proliferation of accounts.

The use of different reporting views, as well as filters and advanced segments is also not very common. This means that Analytics data are just analysed in aggregate, without telling you much about the specific audience you intend to reach. For example, if your website is targeting users in North Africa and the Middle East, you need to be able and single out traffic from these regions, to better analyze your target audience.

Tracking goals and ‘conversions’ is not always common practice. Goals can be set up in various ways in GA to track users’ interaction with the website - when they scroll on the page, click a link, decide to print a page, comment or spend a certain amount of time on the page. This can provide a great deal of information to website managers and editors, to improve the way information is presented and webpages are organized, as well to increase users’ engagement.

Only one out of 6 organizations stood out in using an advanced configuration of Google Analytics to create different reporting views, filter data, track goals and conversions.

Google Analytics too often stands alone 

I have highlighted before the importance of a digital analytics and measurement plan - and how Google Analytics may eventually just be a part (even if the most important) of your data collection and analysis system. On the contrary, I didn’t find a lot of this in the websites I’ve recently reviewed.

On the one hand, while Analytics is the default tool to track digital analytics, in most cases is also the only monitoring tool. On the other hand, when digital analytics are collected from different sources, (e.g. website, newsletter, RSS feeds, social media, etc), more often than not they are not presented and analysed in aggregate. Finally, not all organizations are regularly producing actionable reports on the basis of their analytics, to inform future actions and improvements on the website.

Only one organization presented a more advanced understanding of its’ digital analytics process, with multiple data collection points (e.g. website, newsletter, RSS feeds, social media, etc) that fed into a dashboard spreadsheet, using formulas and calculations to avoid double counting and over-reporting of metrics. Even if there was no document describing a strategy, this is already a great step towards more efficient use of digital analytics.

What can be done? 

I think a lot could be achieved through the availability of more specific content for international development and the open exchange of experiences around digital analytics for the development sector.

The majority of information and guidance available online, while comprehensive, in general tends to focus on e-commerce and more business oriented websites. Other sources such as the Digital Analytics Programme (DAP) provides a good example of guidance and best practices, training and support in digital analytics. However, the target audience is also very specific, DAP being designed for US Government government agencies that provide information to the public. Eventually, there is not much available that focuses specifically on digital analytics for development - and information and knowledge services specifically.

Secondly, I think website administrators and managers should be more open about how they do digital analytics, as ODI has been doing by sharing their M&E dashboard. Knowledge sharing and learning opportunities should be created for users to exchange notes and learn from each other, to identify good practices and examples that can be replicated. Ideally, I believe that web managers should also be open about the actual number of their website stats. Especially for publicly-funded websites, this would mean more transparency and the possibility to compare and benchmark different websites.

Finally, I think donors should play their part in fostering better use of digital analytics in projects and programs they fund. Besides acting as convenor for peer learning initiatives around good use of digital analytics, donors should provide stronger guidance and support in this area, to make uniform data tracking and collection across different projects. Ideally, for donors’ funded websites and knowledge services, there should not just be the mention of few, poorly selected web metrics in the project logframe. A digital analytics and measurement plan should be developed as part of the project inception phase.

In the next post in this series, I’ll look specifically at what metrics and indicators could be most useful, amongst the dozens available, for development websites and knowledge services.

Monday, August 22, 2016

Understanding digital analytics

In the past few months, two different projects gave me the opportunity to spend quite some time working on Google Analytics. The projects were different in scope: in the first, I was asked to review the use of Google Analytics as part of the monitoring and learning system of a think tank; the second project was part of a larger evaluation of development information services, to understand the reach and use of different websites. Overall, I was able to review how 6 different development organizations are currently making use of Google Analytics to track web traffic and users’ interaction.

digital analytics

This is the first post in a three post series where I’ll share some of the learning from this recent work. I’ll start with some key points for efficient use of Google Analytics (GA). Then in the second post I’ll present examples of how different organizations are using GA. Finally, in the third and final post in the series I’ll zoom in more into Google Analytics metrics - and measuring what matters to you.

Google Analytics - from basic to advanced use 

GA has developed a rich and sophisticated toolset over the years. It is now one of the most commonly used tools to monitor website traffic and engagement. It’s probably the industry standard for web analytics across different business domains, such as e-commerce, government, education, and development, too. While some have questioned the accuracy of Google Analytics and there’s no shortage of alternatives tools out there, in my opinion Google Analytics remains one of the most powerful tools (for everyday use, especially in smaller organisations). One of the aspects I like most about Analytics (besides the fact that it’s free…) is its continuous improvement - in terms of its own features and functionality as well as integration with third party tools (such as Supermetrics), and integration with Google Sheets add-ons.


I recognize GA features can be overwhelming: with all the information GA can track it’s easy to drown in a sea of data. Luckily, a simple search in Google will return a lot of results pointing you to tutorials, guides and training videos that will allow you to go well beyond a basic knowledge of Analytics. If you’re new to Google Analytics, or want to take it to the next level, I suggest you take a look at these resources:

As a minimum requirement, your correct installation of Analytics should include:

Analytics measurement plan

It’s relatively straightforward to get your Analytics set up properly and tracking data. But this is just a small part of the job. In fact, even before you get going with Google Analytics (or any other web analytics software), what you need is a measurement plan.

A measurement plan is a document that:
  • Defines your business objectives and outcomes you want to see; 
  • Presents the strategies and tactics to reach these outcomes;
  • Illustrates the metrics you need to monitor and the tools and processes to collect data; and
  • Includes goals and targets for your selected measures. 
In this sense, Google Analytics is only one of the different tools that you have to measure your objectives and outcomes. If you’re active on Facebook and Twitter or publish video on YouTube, these should also find a place in your measurement plan.

There’s a lot of good resources out there on digital analytics and measurement planning. You can check Google’s own guides or this post from analytics specialist Avinash Kaushik. Finally this guide presents a great intro for non-techie on how to create a measurement plan for Google Analytics.

So as essential as a measurement plan may seem, my recent work experience tells me that this is probably more the exception than the norm in development organizations. I’ll discuss more about this in my next blog post. In the meantime, how would you consider your current use of Analytics? Are you using it to its full potential? Do you have a digital measurement plan? Let me know in the comments below!

Thursday, July 21, 2016

Building a knowledge portal through Open Data

The Open Knowledge Hub (OKHub) is a collaborative initiative led by IDS to make good quality research accessible in an original way.

In its essence the OKHub is a “database of open-licensed metadata (bibliographic data and links) about research documents, organisations, and other materials.” Around 20 knowledge partners such as Eldis and 3iE contribute their content to the platform, including titles, URLs, abstracts and summaries, keywords, etc, of the research publications in their catalogues. To date, the OKHub contains over 20767 documents. You can browse and search this wealth of information by different criteria (e.g. themes, languages, regions and countries, etc.) on the Content Explorer.

But collecting, aggregating and organizing this global content is only half of what the OKHub offers. In fact, the OKHub uses the same open infrastructure and technology to allow you to use its content to set up your own knowledge services. Services such as BRIDGE and the Gender Hub are integrating OKHub contents to expand their online collections.

Earlier this year I supported the development of a prototype website that makes use of the OKHub dataset and functionality to presents selected research on Challenges to Development in the Arab World.

Setting up the prototype 

The OKHub offers functionality for developers and site ‘builders’ to re-use its content. You can use a simple HTML widget to display selected resources from the OKHub catalogue. Alternatively, if your website is built on Wordpress or Drupal, you can use a plugin to seamlessly import selected contents from the Hub into your own site.

For the Challenges to Development prototype, we experimented both with solutions. Eventually, as the site is build on Wordpress, we downloaded and installed the OKHub plugin to import around 140 free Open Licensed content items relevant for the 10 key issues covered by the prototype. These contents are aggregated on the resources page and presented separately on each thematic page. 

Together with this open content imported from the OKHub, the proof of concept also provides two spaces for content creation and curation: a section to present featured publications and a blogging space to share relevant highlights from the MENA region.

This project was rather short and straightforward, but there are three key lessons that I think it’s worth sharing.

A business case for the OKHub initiative and platform 

Actually, two. On the one hand, as knowledge producer or intermediary, you can make use of the OKHub technology and infrastructure to contribute the content of your organization, thus increasing its visibility, availability and accessibility. On the other hand, The WP plugin has huge potential, as it allows non programmers to easily import content and augment their own knowledge service, or create a new one.

The human factor

Open content and automation alone are clearly not enough. If you want to maximise the chance of research uptake, the human factor is key. This means using a moderator with the required regional or thematic knowledge for quality control purposes and to tailor imported content to specific stakeholders. But it also means having resources to create your own original content, to curate and repackage existing content, to build and animate a community around your service, to ensure users are interested and engaged.

Tech for the (non-)techie

The HTML widget and WP plugin enable less technical people, with a basic knowledge of HTML and CMS, to “plug-and-play” and build applications which meet their needs. However, you may still need some programming skills, to be able to fully integrate OKHub content on your own site. In my case, there was a conflict between the WP plugin and the site theme, resulting in individual records not fully displaying, or altering the site layout. Thanks to our colleague Tony Murray for stepping in and getting well beyond where my technical knowledge ends!

Overall, the prototype offers a good proof of concept for the idea that open knowledge and collaborative approaches can help extend outreach and uptake of research knowledge.

Do you know other examples on initiative of knowledge services based on Open Data to sharing and use development research content? Let us know in the comments below!

Monday, July 18, 2016

Audit your Google Apps with GAT

For a couple of years, we’ve been supporting the deployment and adoption of a KM platform for the Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA). As the programme has evolved and matured, so has the platform, with almost 500 user accounts, and anecdotal evidence of its usefulness to support knowledge sharing.

But besides counting the user accounts created, what is really happening on the platform? Can we learn more about the users? What are they contributing? Is there any champion emerging? What Apps are most used?

Go beyond Admin Reports 

Google provides its own reporting functionality through the admin panel. If you have a domain admin account, you can access Reports and track Apps usage, security, accounts activity, etc. The reporting features are rather rich and are a perfect fit for ongoing monitoring of the platform. However, Google Reports allows you only to look at data for the previous 6 months period - which is probably not enough if you want a comprehensive picture of how the platform has evolved over time, and what users have been doing with it.

For this purpose, the best solution we could find is the General Audit Tool (GAT).

Launched in 2010, GAT is primarily an auditing and it monitoring tool. It allows you to audit or report on over 250+ separate items for users, documents, email, calendars, sites, groups, etc. Additionally, it counts users’ collaboration activities and calculates a ‘collaboration index’ across your domain, using multiple indicators such as file shares and file visits. Finally, you can set up alerts to get notified if domain policies are not followed - for example, when documents are shared outside the domain. The animated video below provides some more background information on GAT and what it is good for.

GAT comes with a cost, depending on the number of active Google Accounts you have on your domain. However, it also offers a full features trial. If you are using Google Apps, I recommend you test it out and find what it can do for you.

How we used GAT 

GAT helped us to extract a large amount of specific information on users, the frequency they interact with the Apps, and how they work with other users.

We run several daily GAT scans over a period of two weeks and exported several datasets from the Apps and metrics we had decided to include on our analysis. We then loaded this data into Tableau, to be able to aggregate it, segment it, analyze it and make sense of it through charts and tables.

You can read below here some highlights from our analysis:

  • The growth of and demand for new accounts has been steady and well beyond the initial expectations. 
  • The majority of users are active, with 75% of them that logged onto the platform at least once in the past 6 months, and over half of them in the first quarter of this year.
    Date last login
  • The use of the platform has been increasing over time. However, this use is unevenly distributed, with some users clearly emerging as platform champions
  • Google Drive is by far the largest app is terms of usage and the most frequently accessed by users, followed by Calendars and Hangouts. Drive currently hosts over 23K files and folders. The primary function of Drive appears to be to store and archive documents; the creation of new content is secondary. About 50% of all files on shared Drive have been created elsewhere and then uploaded onto Drive. 
  • An increasing number of users are viewing and editing documents on Drive, confirming the adoption of the tool. However, collaboration appears to be limited to a small number of documents, while the great majority see a small number of ‘actions’ (views or edits) performed by an equally small number of users.
    Docs overview: number of users, edits and visits per quarter 
  • In several instances, users are contributing to the CARIAA platform with their personal Google account instead of their CARIAA account. This has potential negative implications in terms of sharing settings, document management and overall platform M&E. 
This is very much a work in progress, we’re learning as we go and constantly testing out new options. What’s good about our progress so far is that we’re generating the kind of longer-term, trend data that really helps us provide support and the client to adapt to evidence about pattern use. And once it has been set-up it is not too time-consuming.

Of course a lot of these features are available in those expensive all-in-one packages used by commercial organisations and the deeper-pocketed big NGOs. But it’s hard work keeping up with trends and providing accurate, useful, timely data on a smaller budget, one more typical of the mass of Development players. So help us - what tools and approaches have you found useful and can share? And are your clients or service users listening to you and the data and changing how they work?

Wednesday, July 06, 2016

Making change happen - KM in a WSH team

It is cheaper and easier to change information flows than it is to change structure.
Donella Meadows

Identifying what causes change in organizations and attempting to identify the impact of specific projects is the kind of conundrum that keeps consultants and academics in profitable and engaging work. To borrow from Outcome Mapping language, it’s a major step to be able to identify whether those people or organizations directly connected to a project, within its’ potential sphere of influence, change their behavior and work differently in ways that could at least be linked to the activities in the project.

Donella Meadows’ seminal work, “Leverage Points – Places to Intervene in a System”, in particular her description of the role of information and feedback loops, was one of the framing ideas for a review workshop of the KM project in the Building Demand for Sanitation (BDS) portfolio. Meadows’ work explores systems, their complexity, and the enormous effort and time required to achieve lasting change. Meadows’ work highlights the importance of power and paradigms, reinforcing the central importance of leadership, a point we’ve made consistently in this series of blogs.

The WSH team of the Bill & Melinda Gates Foundation organized the two-day workshop in September 2015. Its’ purpose was to review 18 months of KM initiatives by the BDS team, as well as Foundation-wide KM experiences, and consider activities for the WSH team as a whole that would lead to stronger networks among foundation and grantees, improve availability and access to specific knowledge, and strengthen the organizational culture, improving the flow of knowledge.

The whole WSH team was involved in the workshop. The BDS KM team shared summary findings from a grantee survey, giving responses on elements of the BDS KM program they valued and whether or how it had affected their work. This graphic below illustrates the relative valued add of program activities, according to grantees.

(click on the graphic for a larger view)

And although 18 months is a short time in which to achieve the more fundamental changes in behavior that are the basis of sustainable change, there were clear indications that grantees believed the BDS KM activities were helping them integrate more effective KM into their work. For example, from the pre-program survey in 2014 we identified grantee KM priorities and in general, in the concluding survey, grantees rated the project’s impact positively. 

(click on the graphic for a larger view)

Mainstreaming KM into the Rhythm of Business

Everyone in the WSH team had ideas and experiences to share, so much so that when it came to prioritize proposals, a senior member of the team responded that he felt almost promiscuous because there was so much that turned him on. It’s hard to summarize such a free-flowing, well-informed and thoughtful conversation but the remarkable graphic facilitation of Nancy White at least conveys some of the richness.

The main theme that emerged was the necessity of integrating KM in the normal ‘Rhythm of Business’ (RoB). There was a consensus that KM has to be ‘mainstreamed’, not seen as something discrete, made up of specific periodic activities. The most fundamental recommendation was that the WSH Director would ask WSH team members to put KM activities into individual goals on basis of common team goals to be developed by management, based on a menu of Key Performance Indicators (KPI) to choose from. This would be supported by including KM in the job description for the then-about-to-be-appointed Deputy Director for Strategy, Planning and Management.

The team agreed also to determine how best to incorporate KM into the grant management cycle, and include it as a standard item on regular ‘Feedback to Action’ meetings. For example, two members of the team planned a pilot of a peer-assist format for part of an upcoming meeting, and they agreed to communicate lessons learned back at the next meeting. Finally, the team planned to institute regular meta-analysis of grant results, one or two times per year, which would feed into the planning process.

Active curation of information and widening access to resources behind paywalls was another theme. The team agreed to put resources towards a service or function that replicated the ‘Curated Updates’ experiment run throughout the BDS KM project. There was also a commitment to exploring how grantees could benefit from Foundation access to publications.

As ever, the longer-term impact of the workshop, and the KM project more generally, will probably be more influenced by the ‘normalization‘ of the concepts through the commitment of so much time to discussion, and the personal engagement of staff in the issue, very much led from the top. The WSH team have committed to reviewing their progress on improving KM, so expect some more blogs in due course.

Meanwhile, what about long-term behavior change in organizations that has demonstrably improved knowledge flows, learning and information management: do you have any examples or ideas?

Friday, June 03, 2016

Learning about Learning - KM in a Gates Foundation WASH portfolio

'O this learning, what a thing it is! '
(William Shakespeare, the Taming of the Shrew) 

It all seems so obvious

Pippa Scott blogged about a conversation with IRC’s Erick Baetings who was convinced he had learnt a lot during the week-long 2015 convening of the Building Demand for Sanitation (BDS) portfolio of grants. But when, with a colleague, he tried to write down what they had learnt – it all seemed rather obvious: ‘we need to think about smarter subsidies‘; ‘we need to work outside our silos.’ Erick said that it took them some time to really work through some of the statements to filter out the real-take home messages (which form an excellent blog here).

Nancy White has played a ‘critical friend’ role with us on the BDS Knowledge Management (KM) project. Nancy and I have worked in KM for more years than we care to (or can) remember. So when we spent some time reflecting what we have learned about learning during the 18 months, a lot of what came up seemed so obvious. But then, we too rarely write down what we know we know, which often means that learning isn’t passed on. That insight came from a shocking moment at the Hanoi convening when it became clear that the same organization had repeated the same errors in different countries over several years, a story told in a previous blog.

The challenges that KM is trying to address don’t change, which is one of the reasons why suggested solutions often sound so obvious: they have probably been tried before, with varying degrees of success. So in this blog we are reflecting on what we have learnt from BDS KM around three themes, identifying what stands out as things that might help improve KM in all our work.

Listening not hearing, observing not seeing

What shakes us out of our comfort zones, makes us challenge our assumptions, makes us recognize that what is in front of us isn’t explained by our current intellectual frameworks? In other words, when and how do we learn?

There are endless quotes on the importance of failure to learning – because failure publicly demonstrates that our assumptions and plans were wrong or inaccurate, and the bigger the failure the harder it is to learn and adjust. Disasters and emergencies have similar, distressing impact on our learning. Merizow suggests transformational learning only happens because of such ‘disorienting dilemmas’.

We can’t wait for failures or disasters to trigger our learning yet often busyness means we miss the obvious and important. The story of how Menstrual Hygiene Management was taken up as an issue by WaterAid GB provides a stark illustration. MHM has emerged as a key concern on WASH agendas in the last 10 years, as the scale, complexity and seriousness of the problems attendant on inadequate menstrual hygiene became clear. Mahon and Fernandes in their seminal 2010 paper, “Menstrual hygiene in South Asia” recount how it started:
“In January 2007, during a project visit to a village in Sehore district of Madhya Pradesh State, an adolescent girl told WaterAid staff that her mother did not allow her to use the household’s toilet during menstruation, because she is impure. During another visit to a village in Sheopur district, a woman casually mentioned in discussion that during menstruation she has used the same set of cloths for the last four years. These two small incidents brought to light another dimension of hygiene, and WaterAid realised that this is an area which has to be addressed”
Enormous kudos to WaterAid for picking up the issue and building momentum in the sector. But what is striking to me as a newcomer to the WASH sector, a man with a daughter, is that such a serious issue emerged so recently. Had the stories told by the two women to WaterAid project officers never been told before, or was it simply that the implications and impact of similar stories hadn’t been noticed? What must we do to ensure we hear and see what is in front of us? And the challenge is greater within projects and organizations, which quickly develop a way of seeing and doing that tends to be reinforced by internal processes driven by the need to meet deadlines, targets or budgets.

Of course sanitation as a sector has the powerful model of Community Led Total Sanitation (CLTS) illustrating how people can be brought to a different way of seeing. And although far less dramatic, consultants, reviewers and evaluators can similarly hold up a mirror to an organisation or project. A similar engagement with outsiders comes from exchange and field visits too, as we discussed in an earlier blog. Both the visitors and the ‘subjects’ challenge each other, stimulate reflection and discussion. Visits to new places and contexts also caters for the random, chaotic, free flowing, emergent nature of many learning processes. Visits trigger experiences and memories that can take participants conceptually to a different point, and are especially powerful if groups or teams are involved.

What have we learnt about learning inside organisations?

But what about internal processes? Somehow teams and projects have to bring the outsider, the ‘other’ into their conversations and reviews, whether literally or through how they approach learning. Someone has to take on the outsider role, be a ‘critical friend’, challenge norms, assumptions and, ‘the way we do things around here’.

It’s obvious that leaders play a crucial role. For example, in the MHM case, the two WaterAid field workers were listened to by their manager, who agreed to take the issue up and invest. In the same way, other managers championed and invested in MHM research and programing within the organization. In the same way, managers and leaders have can profile and model learning, as Jan Willem Rosenboom has done in BDS KM and as did the Directors of the WSH team in the Bill & Melinda Gates Foundation in setting up a two day discussion in Seattle to consider how the BDS KM experiment could inform their plans (the outcome of which we’ll be describing in the next, and final, blog in this series).

Part of that modeling is being open about not knowing, admitting ignorance. By asking genuine questions and sharing their own mistakes and failures leaders can help other people unfreeze, as this great example from a USAID project illustrates.

Nancy described this as “othering.” She suggested that an element of othering is the experience and power closeness or distance between the person sharing a learning and the listener. The closer people are in experience and/or power, the easier it is to share those daily, little things that often matter. The risk is low. Trust is probably high. Reputation is not an issue. Sharing with someone with more power or experience (like a boss, expert or funder), while requiring more courage (or trust), may shine a broader light on an issue that the practitioner may have thought was unremarkable, but the expert discerned its larger importance. So this idea that we need to share knowledge with different kinds of others emerged in our observations. It can also affect mentoring, another knowledge sharing vector.

We also observed it is essential to be explicit and intentional about reflective, learning processes. There are myriad formal processes, for example, evaluations and donor reviews. But in many cases a simpler, basic approach is required in standard meetings, mirroring what happens when disorienting dilemmas or emergencies cause us to stop; to think, “what did I actually learn”; “what’s new”; “what do I know more about now” or “how can I apply this new knowledge”? We have to watch for and catch ourselves and each other in our learning, and be mindful of the larger learning journey of which such smaller exchanges and events are a part. It’s that process we included in the learning exchange program, through our skype calls and email- learning-journals.

But to make that happen, space needs to be set aside. Anywhere we have worked people have complained about not having enough time to stop, think and record their learning.

Of course people do make time informally: we have almost as many stories of the informal ways that people make time to talk, and hence to share and learn – whether it’s the Friday afternoon kick-back, with beer or strong, sweet tea; the long, dusty drive back in the land-cruiser; or the small hours overnight on planes; or the bars or clubs in most CGIAR centers that serve the same crucial social and knowledge exchange function as, for example, the livestock fairs in Zimbabwe that Charles Dhewa and his team are recording. But to take that learning from individual and small group up to the level of teams, projects and organizations requires planning and commitment of resources to regular, formal Learning Reviews, which focus on observation and reflection, adaptation and change.

However, as a respondent noted during a recent organizational KM review, being asked to identify what has been learned at the end of a period tends to generate disconnected, random items, that hover between the operationally detailed and the strategic. Some organizations or program units are addressing this by requiring strategic units to develop learning plans or agendas. Catholic Relief Services, for example, as part of their rich learning program, recently instituted procedures requiring country programs to identify annual learning agendas. The aim is to develop a light-touch frame for the year, areas where the teams expect to know more about, questions that are being explored. 

Learning how to learn, learning how to connect.

Pippa Scott highlighted this issue in her blogs for the KM project post the 2015 BDS convening, asking:
  • “Do people know how to learn in their daily work? 
  • Is it part of their commonly used skills set? 
  • Is there a need to build capacity around learning and sharing within the community? 
  • And do projects and teams take account of people’s learning preferences and styles? 
Learning how to learn can mean on a very practical level, ensuring and supporting community members to know how to use webinar/blog or other online technology. On a more conceptual level, learning how to learn can be to ensure community members know how to slow down and take the time to observe, reflect and learn, as in the 'focused conversation' examples from the 2015 BDS convening. This can also be incorporated into learning events, such as reviews and workshops, where organizers schedule a time at the start of the session to set some ground rules and even learn or practice communication skills, before jumping into the content. For example, during the 2015 convening one group realized that participants needed to tell stories rather than simply throw out all their information at each other. It came to light that one of the participants (Joep Verhagen, WSP) had followed storytelling training. The facilitation team liked this and tweaked a session the next day to include the skill of telling stories rather than simply talking at to each other.

Once we had introduced storytelling as a communication technique, and throughout the rest of the workshop, Pippa Scott noticed that participants would prompt each other into better communication, by asking ‘what is the punchline (of your story)?‘ ‘What do I need to learn from this?’ It was fascinating and especially useful in teasing the golden nuggets of learning from the sheer volume of information. It is often also a more pleasurable experience and people may listen better. "

What works for you? What’s your best example of a project, team or organization that took learning seriously, tracked it, and ensured it fed through into program adaptation and development?

Tuesday, May 24, 2016

What did we learn about connecting people more closely into a global community of practice?

“Any fool can know. The point is to understand.” 
(Albert Enstein)

So think of any large, fun event you’ve ever been part of organizing. You know the ingredients: a good mix of people; good things to eat and drink; some activities – often but not always based around music; a space to gather, preferably one that has lots of different areas, and corners; and you – the hosts, the MC, the facilitators, who watch what’s going on, connecting people who have something in common, who start things moving, mark time and schedule events. And you know when it’s working by the buzz, a mix of different conversations, and the way that people are mixing fluidly.

For managing online communication replace the food and drink with content that people want to consume and the metaphor transfers almost completely.

The ‘connect’ work-stream was central to the Knowledge Management (KM) project we ran during 2014 and 2015 for the Building Demand for Sanitation (BDS) portfolio of the Bill and Melinda Gates Foundation WSH grants. The aims were to:
  • strengthen grantees sense of collective identity, of belonging to a group who could provide support and inspiration; 
  • provide ‘safe’ places to encourage conversation and experiments; 
  • deepen conversations, encouraging double or triple loop learning.  
We’ll use in this blog the components listed above that contribute to successful parties and events as a frame for describing our experiments in strengthening connections between grantees.

Hosting, curating and facilitating – content focused community building

The best party hosts combine social skills with insider knowledge, and enjoy a laugh!. They know who and, often more importantly, who not to introduce to each other. We were lucky to engage Pippa Scott to lead the Connect work-stream, for one day per week over one year. Pippa is a WASH specialist with experience and expertise in facilitating online communities of practice – a rare combination - and has all the social skills of a party host. So Pippa was able to add the role of curator to that of facilitator. As well as seeding and nurturing conversations and connecting different conversation spaces Pippa was able to identify and comment on key content areas, know who to engage on any particular subject, and generate meaningful, specialist content for members of the portfolio to share and discuss. Much of the material in this blog comes from Pippa’s reflective pieces about BDS KM published internally during the project.

Gathering conversations

Targeted and deliberate curation and facilitation contributed significantly to improving peer-to-peer connections within the grantees and fostering a community spirit to enable better knowledge sharing and learning. An indicator is how conversations evolved over time, with face to face meetings spinning off exchanges, which in turn were picked up at the following convening. The BDS community collected a number of conversations, notably:
  1. The Demand-Supply-Finance triangle
  2. Behaviour change, community norms and habit formation
  3. Working at scale – crossing the valley of death from ‘pilot’ to ‘scale’
  4. Learning about learning
  5. The (changing) role of the Gates Foundation
We’ve already blogged about the importance of leadership in modelling effective KM. In the context of BDS KM connect activities, a crucial success factor was having both management and thought leaders prepared to spark the conversations, maintain a strategic perspective and frame ‘knotty problems’ in ways that engage others. Leadership of that kind sets the tone, affirms that not knowing and failing are pre-requisites for learning.

Multiple spaces and interfaces for exchanges

There is no one size fits all when it comes to learning. Everybody learns in different ways and has different learning styles and skills. Some relish a written debate (on email or a forum for example) where others will need more direct or personal engagement. Some are happy to debate in a public space whereas others are not. So it’s crucial to ensure the conversations take place across a number of platforms where each conversation creates an interface or opportunity for the community’s connections to be reinforced.

As described in an earlier blog, face to face, voice and email are the communication preferences for BDS grantees –which our experience elsewhere confirms is typical of the Development sector. Our challenge was to link and build connections between the face-to-face events, such as the the BDS annual convenings and the round of WASH conferences, workshops and events in which grantees otherwise crossed over. So we used a mix of online platforms, illustrated below:
  • An email list as the primary communication channel (using; 
  • A private blogging space (using to help the community protect their learning space, and where we shared other information about projects and grantees
  • Social media, particularly, to link with the small but growing band of digitally-active grantees 
  • Webinars, both private to the portfolio and public, via

Unsurprisingly, the email list was the most heavily used. But there was also moderate and growing use of the private space, particularly the blogs, as the conversations described above rippled across the platforms. A key web indicator is average length of time users spend on a page. The vast majority of web pages score under 10 or 20 seconds, so the two-minute average for the BDS sites was encouraging. The blogs also had a lower bounce rate (people who leave the site after visiting only one page).

Activities and Learning events

The currency of online communication is content and events. So we planned a series of activities including targeted questions, reflections and reports from exchange visits, and webinars following up from the face-to-face events. We anticipated that each would attract overlapping but different audiences. We maintained deliberately a low-level of regular communication, with the curated updates service as a steady drip of targeted content to maintain and grow interest.

We wanted to identify the hottest topics for grantees, what people are grappling with daily, what issues had the greatest potential for exchange. The open agenda calls initiated by Jan Willem Rosenboom, the BDS portfolio lead, were described in the second blog in this series. Their purpose was to provide a forum to connect outside of the annual face to face meetings and share sector updates, not just issues relating directly to the BDS portfolio. The evolution of conversations within and around those calls illustrates the role of small connect investments. The first round of calls was rather functional, where several organizations voiced an area of interest where they could offer or would appreciated some peer support or insights from others experience. The calls were evidently beneficial to grantees and sparked several one-to-one offline conversations for peer-to-peer exchange immediately after.

The topics raised during this first round of calls (financing vs. demand, learning about learning and monitoring platforms) informed the early conversations within the BDS KM activities and these broad themes have since flourished (having been nurtured with support of a series of KM learning events and activities) into an informed and quality discourse. The content of the second round call was more focused on sharing learning, with grantees identifying possible synergies of their work (as opposed to general assistance requests) with a deeper quality dialogue than the first round of calls 7 months earlier. In general, the conversations amongst BDS grantees became much more focused and nuanced in their discourse over the program year. Pippa Scott’s view is that it is through allowing these conversations to flow, through the community, picking up different aspects but maintaining a steady and focused flow through different platforms and gaining insights from different people (professionals, practitioners, academics) that such a rich “collection of conversations” emerged within the BDS network in a relatively short space of time.

Consumable content

Following the 2015 face to face Sanitation Partners meeting in Hanoi, the reflections of Gates Foundation staff and the BDS KM team were that the annual BDS face-to-face convenings really do provide a forum for state of the art discourse to be voiced and shared. Where others in the sector may be waking up to potential synergies of programs, the BDS Sanitation Partners forum actively brought partners working at the forefront of rural sanitation together to exchange and learn from each other.

The challenge for the BDS KM team was to try and maintain some focus and quality to these conversations outside the face-to-face events. As such, the BDS KM team, responding to the feedback of participants, attempted to channel and foster the conversations through a series of online learning events and resources. The most notable of which were: blogs following up from Hanoi, thematic webinars on issues raised by BDS grantees (recorded and shared within, the learning exchange visits described in the previous blog (shared with grantees on email and in summary blogs on and in certain cases one to one exchanges of BDS KM staff with BDS grantees.

The right people

Samoan Circle discussion at 2015 BDS Convening
The potential for useful exchange and learning within such a diverse group as BDS grantees was a key driver for the program, especially since our surveys showed that grantees under-valued themselves as a source of knowledge and learning, even though the portfolio brings together many of the leading organizations in Sanitation, including acknowledged thought leaders. 

Does the BDS buzz represent a positive return on investment?

Too much communication becomes noise, too little and the level of communication between face-to-face events drops to near zero, as was the case in BDS before the KM project. As we’ve described above, we aimed to provide just-enough communication, initiate activities that would attract grantees because they were interesting and relevant, while weaving content and conversation between different channels and face-to-face meetings. And, within the narrow bounds of this 18 month experimental project, our review showed that there was indeed some change in behavior, as illustrated below.

Fig 1 How grantees communicate between annual meetings 
The overall level of investment in the connect activities, including both Pippa Scott’s one day per week and contributions from other BDS KM team members, was approx. 30% of an FTE. We would argue the level of engagement and changes in behavior among grantees represents a positive return.

Do you have other examples of similar targeted KM investments in programs bringing a range of organizations together in a relatively loose association such as in the BDS?