With the world’s thought leaders in innovative policymaking having taken to the stage of the Policy Innovation Exchange, several cutting-edge and useful innovations were discussed during Thailand’s first event of its kind. Among the ranks of the experimentation experts present, Calum Handforth, Thematic Lead for the UNDP Global Centre, Singapore, took to the stage to provide an overview of data analytics, and how data can be leveraged to enhance policymaking processes.
Background & Premise
In a world where “data is everywhere”, many sectors, including the government, are experiencing pressures to move their services and attention to online arenas. With users beginning to wonder, “why is it harder to get a driver’s license than to create a Facebook account?” the bar for standards of service delivery has been irrevocably raised. As a result, more than ever before, digital channels and strategies are now considered to be key development tools, as they provide new channels to deliver government services and engage civil society. In moving to digital, user data is generated from which insights can be extracted by leveraging cutting-edge data analysis techniques in the form of collective intelligence, big data, and even applications of AI, to name a few.
Transitions to Digital Analytics
According to Calum, the world was forced to rapidly transition due to the COVID-19 pandemic, where “more than two years’ worth of digital transformations have taken place in the last two months.” This has led to unprecedented high rates of data generation, and digital technology adoption rates soaring. While this change may have been met with some resistance, Calum summarizes our collective experiences in saying, “COVID-19 really highlighted that digital transformation is not optional for governments. It’s fundamental to service delivery, engagement, and it enables things like remote working and remote learning.” Yet, having reached nearly oversaturated levels of digital transformation, Calum posits that all sectors, and specifically policymakers find themselves in an ideal position to explore new ways of leveraging digital tools for national development, policy development, growth, and service delivery.
Digital Analytics as Measures of Success
This prompts the question: how best does one leverage digital tools for these purposes? As the digital world hosts more data than ever before, and sees new digital tools being created at unprecedented rates, measures of success in digital settings are required, and this is the premise behind digital analytics—determining, qualifying, and quantifying successful outcomes in the world of digital. This success is determined, broadly, in 4 main categories of data obtained from digital platforms in both active and passive digital analytics, namely, lifestyle analytics, activity analytics, and engagement analytics.
- Lifestyle Analytics
According to Calum, “Understanding the lifestyles of users is critical to disseminating messages to targeted audiences.” Here, analytics are performed on primarily socio-demographic data gathered by software or online platforms, e.g., Facebook, Twitter, or Linkedin, and comprise common data points such as age, gender, occupation, location, as well as affinity and interest categories from online targeted ads. Creators of content and messages for large swathes of populations curate and display highly tailored messages for those who engage their platforms based on these variables, and this allows for very granular differentiation between groups. Calum also states that such lifestyle analytics should be understood to be representative of online lifestyles but respresentation of users is important to consider, and that the same data from the same person are often confounded when met with offline lifestyle—there may be a disconnect between these two realms, and the transitions and connections between them are still unclear.
- Activity Analytics
With increased use of digital media and the corresponding data which is collected through interaction with these tools, we can extract insights through statistical methods from a number of sources. Such sources can include, exploration of user journeys, link clicks, mouse movement (click streaming), the number of users at one time, or chatbots, where even seemingly humble variables like timestamps can be used to better understand user interactions online. Calum also speaks to the importance of this form of data during COVID-19, where a mass of such data was collected concerning lockdown policies, GPS-data of people’s geographic movements. As with all forms of data analytics, Calum warns of the importance of validating and updating one’s assumptions and hypotheses, as in his experience, what one thinks may be important, may not be important as something which was potentially overlooked. A similar possible pitfall to consider when dealing with activity data is that often such data are owned by the private service provider whose platforms provides the data.
- Engagement Analytics
Engagement analytics represent a deeper dive into the types of data which can be collected from user engagements with a given platform of service delivery method. Engagement data can be taken from chat logs derived from chatbots, activity point data, records of posts, forum discussion, and others. As these data are often related to human language, useful statistical models which analyse and classify such data can require some advanced techniques to break down the info and produce valuable insights through methods such as Natural Language Processing (NLP), or sentiment analysis.
- Outcome Analytics
With outcome analytics, developers, analysts, and policymakers unlock insights which allows stakeholders to better engage how a tool or platform impacts on people’s lives.
While there are any number of tests which can be performed to extract insight in this area of analytics, as it is an emerging field, Calum exemplifies its use through use of a trending method known as A/B testing. With A/B testing, operators are able to measure the effect of changes or movement of objects within a given platforms, e.g., move buttons to new positions and monitoring the new vs old position through usage data. Another way in which A/B testing is used is to test colours of elements in a piece of online literature, or even simple buttons. Here, click rates can be measured and used to support layouts, design elements, or even elements of content in order to determine what produces the best outcomes with users.
Regardless of which field of data analytics policy happens to pull from, Calum notes that there have been rapid developments across nearly all sectors in recent times which allow analysts and, to some extent, non-analysts alike to come closer to insights. For instance, Facebook now allows users to view a significant amount of backend data and provides insights into user behaviour. In addition, there are many out-of-the-box ready solutions, such as chatbots, education portals and others. Perhaps most intriguing is that many smartphones today contain a large number of sensors, which can be leveraged to provide end users with insights such as the fastest route to work, or which exercise can assist them in achieving their weight-loss goals.
Case Studies & Projects
Calum, himself, has extensive experience in dealing with new forms of data, which stems from his work in the health literacy programs prior to his time at the UNDP. During this program, which saw deployment of a chatbot across five countries in the Asia-Pacific and Sub-Saharan regions of the world so that adolescents were empowered to make more informed decisions about health and relationships. Based on deep understanding obtained from several engagement campaigns with their target audiences, Facebook Messenger was chosen as the appropriate vehicle to host the chatbot, and within a very short time more than 2,500 users had been amassed. Given the wealth of data which was obtained, Calum and his team decided to investigate how exactly users are engaging with the chatbot in order to adjust services and increase user-platform fidelity. Based on their research, they found more than 1,600 unique user journeys, and insights which provided a glimpse into the lives of users, both online and offline, and their well-being.
Data Analytics as an Empowering Tool, Not the Final Answer
Yet, the extent to which digital analytics can be employed to better engage an audience or more closely approach them without the need of intrusive or extractive data collection methods does not stop here. Calum continued to mention still two other methods employed by the UNDP, which have resulted in enriched understanding of people and therefore contributed to the overall approach adopted by policies, which either actively engaged users through expedited data collection methods, or passively engaged them, which in Calum’s views produces more consistent and better-quality results.
While the world of data analytics may seem robust enough to tackle most any questions in a digital space, Calum is quick to warn that in most all cases “It is not sufficient to rely on technology alone. We need expertise and domain knowledge to make sense of the data collected using our technology.” Put simply, data analytics is a tool which stands to enhance, not replace, policymaking procedures and should be integrated into current policymaking frameworks as such. According to Calum, “Human ingenuity, expertise, and knowledge should always come first,” and digital technologies act as amplifiers or extensions of this knowledge to allow for more robust decision making, while also addressing several problems relating to efficiency.
