Open-ended survey inquiries to elicit individuals’s first-order considerations

The general public debate is dominated by controversial and polemic points in politics, science, and well being. With rising proof that Individuals maintain strongly polarised views about insurance policies (Nyhan 2020, Alesina et al. 2020), it’s changing into of vital significance for researchers and policymakers to develop new and rigorous strategies to take heed to residents of various backgrounds and to raised perceive their views. 

Surveys: A singular analysis device 

To this finish, surveys are a novel device to grasp what is occurring in society and in individuals’s minds – what they assume, how they cause, and what their considerations are. Few different analysis instruments permit researchers to take such a snapshot of individuals’s minds. When designed nicely, calibrated correctly, and focused to related samples of curiosity, surveys allow researchers to conduct on-line, large-scale investigations in real-time. They’re intuitive, clear, and adaptive to members.

Surveys have up to now helped make clear many related coverage matters. One notably fruitful line of analysis has leveraged survey knowledge to discover individuals’s perceptions and preferences about tax coverage and redistribution (Cruces et al. 2013, Karadja et al. 2017, Alesina et al. 2018, Roth and Wohlfart 2018, Fisman and Kuziemko 2020, Hvidberg et al. 2021). 

The worth of open-ended survey questions

The spine of surveys typically consists of closed-ended questions that present a hard and fast set of reply choices. The benefits of these questions are that the reply choices are standardised and streamlined throughout respondents they usually simply lend themselves to quantitative evaluation. Nonetheless, in some settings, we might unintentionally prime respondents to consider reply choices that they’d in any other case not have considered. Conversely, we might omit related choices that we have no idea about. In open-ended questions, respondents are usually not supplied reply choices, however relatively an empty textual content entry discipline through which they’ll write freely. Open-ended survey questions can subsequently circumvent a few of the above-mentioned points. By being much less guided, they could educate us issues that we might in any other case have missed and that we might not be used to enthusiastic about. On this sense, solutions to open-ended questions might assist to make clear the first-order issues that come to individuals’s minds with out constraining them to consider a restricted set of reply choices.

What are some greatest practices when designing open-ended questions? Open-ended questions can vary from very broad to slim and particular. Broader open-ended questions are helpful to elicit first-order, intrinsic considerations that individuals have earlier than they’re prompted to consider a specific coverage side with extra directed questions. Thus, it is sensible to begin by asking individuals massive image questions such because the ‘predominant issues’ that come to their minds when they consider a difficulty (e.g. earnings or property taxes). It’s then helpful to slim down the main target by asking individuals what they consider particular elements of a coverage, its benefits and drawbacks, or much more focused questions on its penalties. Ideally, open-ended questions needs to be complemented with closed-ended questions for cross-validation.

Solutions to open-ended questions might seize two various things. First, the solutions of respondents who haven’t beforehand thought concerning the matter could also be ‘intestine reactions’. These reactions are informative, as they replicate what a respondent thinks and can maintain considering, absent extra studying or targeted reflection. Second, the solutions of respondents who’ve already thought concerning the matter or who take time to consider it in the course of the survey earlier than answering might replicate extra profound views. Both means, solutions to open-ended questions seize the first-order issues that matter to individuals and the elements of a difficulty which might be on the high of their minds.

Strategies of research

In a latest venture (Ferrario and Stantcheva 2022), we apply these open-ended query strategies to knowledge from two surveys on earnings and property taxes performed in 2019 on massive and consultant samples of US residents aged 18 to 70.

We deal with solutions to the broad query, “What are your predominant issues?”, when enthusiastic about earnings or property taxes, respectively, and we suggest three totally different strategies of research. 

Phrase clouds are greatest used as a primary step in visualising the information and for scanning solutions shortly. Figures 1 and a couple of present the phrase clouds derived from the responses. For earnings taxes, respondents specific disagreement with the present ranges of taxes, views on the path through which to alter them (“decrease tax” or “increased tax,” relying on the teams they discuss with), and concern concerning the impacts on the “center class.” For property taxes, respondents’ first-order response centres across the concern of “double taxation” and the equity of property taxes, adopted intently by considerations concerning the “center class” and “household.” 

Determine 1 Primary issues about earnings and property taxes

a) Earnings tax

b) Property tax

            

Notes: Phrase clouds based mostly on solutions to open-ended questions on respondents’ predominant issues on earnings and property taxes.

Determine 2 Primary matters respondents take into consideration concerning the earnings and property taxes

a) Earnings tax

b) Property tax

Notes: The determine exhibits the distribution of matters talked about within the solutions about the principle issues on earnings and property taxes. The bars signify the variety of occasions a subject was talked about out of the full mentions of any matter by political group.

It’s then notably fascinating to check whether or not individuals belonging to totally different teams (e.g. of various political affiliations, ages, and incomes) use a systematically totally different lexicon. We focus our consideration on the heterogeneity by political affiliation and reduce our pattern utilizing a mix of the presidential candidate supported in 2016 (Clinton versus Trump) and of the self-reported diploma of conservatism versus liberalism.

We contemplate two methods of investigating heterogeneity: keyness evaluation and matter evaluation. Keyness evaluation relies on a relative frequency evaluation that compares the usage of phrases between two teams of individuals. This methodology attaches a rating to every time period, which measures how attribute the time period is of 1 group relative to the opposite. Matter evaluation relies on a keywords-count mannequin, the place matters are outlined by units of key phrases. 

First-order considerations about earnings and property taxes

For earnings tax, we establish eight distinct matters: Distribution, Equity, Authorities Spending, Social Security, Effectivity, Loopholes, Flat Tax, Public Items, and Do not Know. The Distribution matter, for instance, incorporates key phrases reminiscent of “center class”, “low earnings,” and “millionaire.” The Equity concern incorporates the phrases “truthful” and “unfair.” Public Items captures “infrastructure”, “training”, and “well being care.” Effectivity is represented by phrases reminiscent of “damage financial system”, “work much less”, “competitors”, and “spend much less”, amongst others. The ultimate class is for respondents who specific that they have no idea sufficient concerning the coverage to offer a significant reply.

The matters of Distribution and Equity, in addition to Authorities Spending and Loopholes, are on the high of many individuals’s minds. The Effectivity matter doesn’t look like as salient. These outcomes echo Stantcheva (2021), who finds that distributional and equity issues dominate effectivity considerations in shaping individuals’s tax coverage views. 

For property tax in Panel b, the matters are much like these from the earnings tax survey. Particular to property tax is the Double Tax matter, captured by key phrases reminiscent of “already taxed”, and the Grieve matter, which is captured by phrases reminiscent of “grieve”, “bury”, and “funeral”. The distribution of matters is once more closely centred round problems with Distribution and Equity, however Double Tax can be necessary. A lot rarer are mentions of Authorities Spending, Effectivity, Loopholes, Grieve, and Public Items. Moreover, extra individuals specific a lack of understanding about property tax than earnings tax.

Partisan variations in first-order considerations

There are clear political variations within the matters talked about. Points associated to Distribution are far more prevalent on the left aspect of the political spectrum than on the suitable, whereas Authorities Spending is a extra urgent concern for respondents on the suitable. Effectivity just isn’t steadily talked about by any political group. Equity mentions are fairly evenly distributed throughout the political spectrum. Nonetheless, despite the fact that everybody cares about equity, the that means of this idea vastly differs throughout respondents; equity is within the eye of the beholder.

Determine 3 exhibits the key phrases which might be most particular to Democrats and Republicans. According to the subject distribution, the key phrases on the Democrat aspect centre round problems with Distribution (“poor wealthy,” “decrease class,” “center class,” and “tax rich”). Republicans have a tendency to stress Authorities Spending, Authorities Waste, and the Financial Prices (“individuals work”, “financial system spend”). For property tax, there are additionally clear variations within the matters talked about by political leanings. Distribution points are most prevalent amongst Clinton liberals and diminish quickly and monotonically in direction of Trump conservatives. Conversely, Double Taxation considerations are prevalent amongst Trump conservatives, however fairly uncommon amongst Clinton liberals. The mentions of Equity are once more evenly distributed throughout the political spectrum. These patterns are confirmed by the keyness evaluation in Panel b of Determine 3.

Determine 3 Key phrases talked about by democrats and republicans concerning earnings and property taxes

a) Property tax

b) Earnings tax

Notes: The determine exhibits key phrases amongst Democrats and Republicans in solutions to the query about respondents’ predominant issues on the earnings and property taxes. The rating reported for a set of two phrases is the χ^2-test statistic, testing the null speculation that the prevalence of the given key phrases is similar amongst Democrats and Republicans. * p<0.1, ** p<0.05, *** p<0.01.

Open-ended questions as a information to in-depth analysis

The solutions to open-ended questions can information subsequent analysis. Within the case of earnings and property taxes, they reveal that individuals care concerning the distributional impacts of taxes and equity. This prompts us to dig deeper into these points utilizing detailed (closed-ended) survey questions and experiments to raised perceive these considerations (Stantcheva 2021). Many analysis areas in economics may benefit from the usage of open-ended survey inquiries to elicit individuals’s first-order considerations and information extra in-depth analysis. 

References

Alesina, A, A Miano, and S Stantcheva (2020), “The Polarization of Actuality”, AEA Papers and Proceedings 110: 324-28.

Alesina, A, S Stantcheva, and E Teso (2018), “Intergenerational mobility and preferences for redistribution”, American Financial Assessment 108(2): 521-554.

Cruces, G, R Perez-Truglia, and M Tetaz (2013), “Biased Perceptions of Earnings Distribution and Preferences for Redistribution: Proof from a Survey Experiment”, Journal of Public Economics 98(C): 100-112.

Ferrario, B and S Stantcheva (2022), “Eliciting Individuals’s First-Order Issues: Textual content Evaluation of Open-Ended Survey Questions”, AEA Papers and Proceedings 112: 1-8.

Fisman, R, Ok Gladstone, I Kuziemko, and S Naidu (2020), “Do Individuals Need to Tax Wealth? Proof from On-line Surveys”, Journal of Public Economics 188: 104207.

Hvidberg, Ok, C Kreiner, and S Stantcheva (2020), “Social Place and Equity Views”, NBER Working Paper 28099.

Karadja, M, J Mollerstrom, and D Seim (2017), “Richer (and Holier) Than Thou? The Impact of Relative Earnings Enhancements on Demand for Redistribution”, Assessment of Economics and Statistics 99(2): 201-212.

Nyhan, B (2020), “Info and Myths about Misperceptions”, Journal of Financial Views 34(3): 220-36.

Roth, C and J Wohlfart (2018), “Skilled Inequality and Preferences for Redistribution”, Journal of Public Economics 167: 251-262.

Stantcheva, S (2021), “Understanding Tax Coverage: How Do Individuals Purpose?”, The Quarterly Journal of Economics 136(4): 2309-2369.