How-to guides for beginners, staff and technical specialists in the social system and related areas

Te Atatū: Developing an indicator of disability

November 2022

The Social Wellbeing Agency has worked with experts in Disabled People’s Organisations and in government to create a new proxy indicator based on government administrative data that can be used to explore outcomes of disabled people. This report summarises the work to create an indicator that identifies people who are likely to be disabled.

This work is part of the Disability Data and Evidence Working Group, chaired by Whaikaha, which is working on ways to improve the collection of data and statistics on disabled people.


Te-Atatu-Developing-an-indicator-of-disability.pdf [PDF, 368 KB]

Te-Atatu-Developing-an-indicator-of-disability.docx [DOCX, 226 KB]


Manawa - key findings

November 2022

This is a summary report of the learnings and findings of the Manawa project, that SWA commissioned Allen and Clarke to write. The goal of this project was to investigate whether a technology-based tool could provide whanāu with a quick and simple way to provide feedback to providers about their wellbeing. The project was impacted by COVID-19 but resulted in some exploratory work with whanāu on wellbeing measures.

Te Manawa Key Findings Final web version v2 [PDF, 10 MB] 



Summarise, confidentialise, and checking tools (guidance)

October 2022

The Social Wellbeing Agency has developed new tools to summarise, confidentialise, and check research outputs from the Integrated Data Infrastructure (IDI). These tools support efficient processing of tasks that are common to many projects: producing summary cross-tables with counts and totals.

By automating manual process with well tested code, these new tools help our staff deliver higher quality work faster and are now available for other researchers to use. Documentation for these tools is available below, and the code can be found with the Dataset Assembly Tool. These tools are also demonstrated as part of the Agency’s exemplar project.


Summarise and Confidentialise Training Guide [PDF, 2 MB]

Self-checking Tools Training Guide  [PDF, 1.8 MB]

Tools available with assembly tool on GitHub. (external link)

IDI Exemplar Project (guidance)

April 2022

This  guide has been produced as an exemplar, to provide new or developing researchers using the Integrated Database Infrastructure (IDI) with a simple end-to-end project from which to develop their own research projects. Our team of data scientists use the IDI every day in our work on challenging social-sector problems, and we created this exemplar to improve the effectiveness with which we deliver analytic projects. We hope it will be useful to others. Producing this guidance also supports the Social Wellbeing Agency's three year strategy [JPG, 60 KB] which includes a commitment to supporting other agencies to improve their data capability.


IDI Exemplar Project Guide [PDF, 490 KB]

Exemplar GitHub Page(external link)


Ngā Tapuae technical guidance

April 2021


This report is a technical guide that describes the quantitative analysis techniques used in a mixed methods research project to identify the most important barriers, levers and boosters to help young Māori to succeed. It also includes guidance on the methodology, and directions on how to use the codes. Our agency led the quantitative phase, using the previously published representative timelines methodology. This work generates new questions and insights, but does not imply causality or significance.

Success frameworks for Place-Based Initiatives: Design and toolkit

January 2021


This report presents success frameworks for Manaaki Tairāwhiti and the South Auckland Social Wellbeing Board (SASWB) as a starting point for future analysis of the ongoing value of these two Place-Based Initiatives (PBIs) or similar initiatives. In particular, the toolkit includes case study templates. Using these templates to draft a case study for each PBI helped test and refine the success frameworks while also trialling a way of demonstrating a PBI’s contribution to social sector system change/improvement.

Accelerating dataset assembly

December 2020


The Social Wellbeing Agency has developed the Dataset Assembly Tool to standardise and automate dataset assembly. Data preparation for analytic and research projects increases in complexity as the number of data sources increases. Without a consistent method, it can become time-consuming, expensive, and error prone. The tool helps staff deliver higher quality work faster and is now available for other researchers to use.

These documents introduce the tool and cover everything a researcher needs to begin using it. The primer gives an overview of the tool, describes some effective patterns and workflow when using it, and concludes with a worked example. The training presentation covers how to setup the tool, how to prepare and assemble data with the tool, and extensive notes on what to do if something goes wrong.

Constructing households from linked administrative data

April 2020


Good household information in linked administrative data is highly desirable because of the significant links between individuals’ experiences and the experiences of other people in their household. We attempted to improve the quality of address information in the IDI by applying a series of detailed cleaning rules to remove patterns of incorrect or untrusted information.

Representative timelines - modelling people's life experiences

November 2019


Representative timelines bring together different aspects of people’s experiences into one consistent view — enabling us to see life events, and interactions between events, over time. They can help determine what questions might help us arrive at a better understanding of people’s life experiences, and their interactions with social services.

This document provides guidance on the construction of representative timelines. Examples drawn from the first application of this work — a study of South Auckland families’ experiences around the birth of a child — illustrate the technique in practice.

Using integrated data to understand mental health and addiction conditions

August 2019


To support analysis of mental health and addiction in New Zealand, we’ve developed a tool that consolidates five key data sources in the Integrated Data Infrastructure (IDI) into a single table of events. The consolidated table applies consistent metrics so researchers can more readily analyse data about mental health and addiction conditions. This technical guide discusses data sources, the consolidated table and what sorts of research questions the new table can help answer. The guide complements the code required to generate the consolidated table, available at SWA’s GitHub page(external link).

An introduction to social investment Analytics

December 2017


This introduction outlines the methodologies we use to turn data into insight in plain english. It explains what data analysts will need to perform social investment analysis, and where to find it.

Beginner's Guide to the IDI

December 2017


Getting started with the IDI is a steep learning curve; we've written this beginner's guide for anyone undertaking their first social investment analysis using this resource. It draws from our experience and covers the things we wish we'd known at the start.

Social Investment Analytical Layer Code (SIAL) User Guide

December 2017


The Social Investment Analytical Layer code creates events-structured tables in a consistent format, making it easier and faster to use the Integrated Data Infrastructure (IDI). This brief user guide contains the introductory information you will need to get started.

Social Investment Data Foundation Code (SIDF) User Guide

December 2017


The Social Investment Data Foundation code produces analysis-ready datasets within the Integrated Data Infrastructure (IDI), reducing the time to generate datasets and speeding up analysis.  This brief user guide contains the introductory information you will need to get started.

Working principles for social investment analytics

December 2017


It’s not just about what we do, it’s about how we do it. This guide outlines workflow practices our analysts have adopted to generate maximum insight in the most efficient way.

How to produce a social investment evidence brief

June 2017


Outlines a process for gathering the relevant evidence to support social investment. It explains how to summarise the best evidence and write it up in a way that will help inform decision-makers.