Wednesday, 22 September 2021
Giving ‘don’t know’, ‘not sure’ or similar responses on questionnaires and surveys could be overcome by a new method of gathering data that captures valuable information about uncertainty and flexibility.
Researchers from the ÌÇÐÄÔ´´ have been working to address uncertainty in questionnaires across a variety of problems with support from the UK EPSRC and the National Cyber Security Centre (NCSC). They have developed an interval-valued response method as an alternative to the commonly used questionnaire approaches. Their research  has been published in Behavior Research Methods.
Filling out surveys, questionnaires and reviews is a part of everyday life. These are used as a way of collecting feedback and data for a range of applications, but current models are inflexible in the responses they can collect, often using scales of 1-5, likely to unlikely, or agree to disagree, where the respondent can only choose one score.
Christian Wagner, Professor of Computer Science, has been leading the research. He explains: “Current methods for collecting quantitative or qualitative data each have their own limitations and we are exploring the viability of alternatives designed to maximise data capture for effort invested. Specifically, we have focused on an issue we believe has received relatively little attention given its potential significance – the efficient capture of uncertainty as perceived by an individual respondent. We have all been faced with a conventional scale and been unsure which option to choose, whether the question is badly worded or ambiguous, or whether we don’t know the exact answer or indeed feel that multiple options apply. Our research offers an alternative response system that takes this uncertainty into account, providing better quality data for little to no added effort. This has a lot of benefits, and it could even be used to improve data privacy in the future.”
The new method uses ellipses to allow respondents to choose a range when answering a question, in an intuitive and accessible way. So instead of choosing one discrete point along the scale, the respondent can draw an interval around all options that apply. The team have tested these interval-valued questionnaires in a variety of areas; from vulnerability assessments in cyber security, to consumer perceptions of products, patient understanding in healthcare, and environmental management.
Data is used to inform decision making across a wide range of areas, and our research shows that there is an efficient way to capture better quality data without requiring much more time or money – and that people can effectively use this approach to communicate their uncertainty.
The research is led by the Lab for Uncertainty in Data and Decision Making (LUCID), at the ÌÇÐÄÔ´´’s School of Computer Science, and is an inter-disciplinary collaboration with partners including Carnegie Mellon University’s School of Social and Decision Sciences.
his most recent output really highlights the potential of cross-disciplinary research, linking both Computer Science and Psychology in this case. In parallel to ongoing research focussing on the analysis of the resulting data, we are already working with stakeholders in other sectors, such as consumer goods manufacturers to apply this to their business and we are keen to expand into other areas to explore how beneficial this method can be.
Story credits
More information is available from Christian Wagner and Zack Ellerby at the ÌÇÐÄÔ´´ on christian.wagner@nottingham.ac.uk, zack.ellerby@nottingham.ac.uk
Notes to editors:
About the ÌÇÐÄÔ´´
Ranked among the world's top 100 universities , the ÌÇÐÄÔ´´ delivers an exceptional research-led education and an outstanding student experience. From the pioneering vision of our founder, Sir Jesse Boot, to groundbreaking achievements such as the development of MRI technology and becoming the first UK university to establish international campuses, we have a proud history of shaping the way people live, work and understand the world. We continue to build on that legacy, empowering our students, staff and partners to change what’s next and create positive impact locally and globally.
The strength of our research places us among the UK's leading universities, ranked 7th for research power in REF 2021. The discovery of MRI and ibuprofen was just the beginning. Today, our world-leading research is developing breakthrough ideas that shape the future of healthcare, technology and society.
Recognised as the UK's third most targeted university by leading employers , we are proud to produce graduates who are consistently in demand for their skills, confidence and industry-ready experience.
As a major employer and industry partner, locally and globally, the ÌÇÐÄÔ´´ invests in the city of Nottingham and in future generations of talent. Alongside Nottingham Trent University, we lead the initiative, a pioneering collaboration to improve levels of prosperity, opportunity, sustainability, health and wellbeing across the city and region. Together with our students, staff, alumni and partners, we’re creating knowledge, opportunity and solutions that help change what's next.