Une­qual par­ti­ci­pa­tion rates in sur­veys or false respon­ses can distort data about voting beha­viour. This can create a pro­ble­ma­tic bias that should be addres­sed with com­ple­men­tary methods.

Die­sen Arti­kel in deut­scher Spra­che lesen.


fter every elec­tion, it’s the same story: poli­ti­cal experts ana­lyse, lead can­di­da­tes explain, and jour­na­lists report. Voter migra­tion, suc­cessful issue framing, and coali­tion opti­ons are dis­cus­sed. For some, the days fol­lo­wing the elec­tion deter­mine the course of their care­ers, while for others, it deter­mi­nes whe­ther they will soon have to adapt to fas­ter depor­ta­ti­ons or a fur­ther increase in air pol­lu­tion. To achieve a com­pre­hen­sive con­tex­tua­liza­tion of elec­tion results and a well-foun­ded dis­cus­sion, „good“ data on the deve­lo­p­ment of voter turn­out, appr­oval ratings of cur­rent poli­cies, or the eva­lua­tion of can­di­da­tes by voters is needed.

„No response“ as an essential part of surveys

During the coll­ec­tion of this data, it should be clear to respond­ents which response opti­ons they can choose from. Even the mere act of rea­ding the dif­fe­rent opti­ons could influence respond­ents by trig­ge­ring cer­tain memo­ries (Diek­mann, 2007, p. 446 ff.). This is par­ti­cu­larly true for quan­ti­ta­tive sur­veys where respond­ents are asked clo­sed-ended ques­ti­ons, such as: „Which party did you vote for in the 2017 fede­ral elec­tion?“. In such cases, respond­ents should be pro­vi­ded with a list of all the par­ties that ran in the 2017 fede­ral elec­tion as ans­wer opti­ons. Addi­tio­nally, to account for unfo­re­seen response pos­si­bi­li­ties, it may be useful to pro­vide an „I pre­fer not to answer”-option for those who do not want or can­not pro­vide a response (e.g., because they do not remember).

The author

Clara Wei­ßen­fels is PhD stu­dent at the pro­gram „Poli­ti­cal Eco­nomy of Ine­qua­lity“. Her rese­arch focus: poli­ti­cal par­ti­ci­pa­tion and power relations.

Systematic Withholding or Just Bad Luck?

Deal­ing appro­pria­tely with metho­do­lo­gi­cal chal­lenges in cap­tu­ring poli­ti­cal beha­viour or atti­tu­des has been a focus of many dedi­ca­ted rese­ar­chers for deca­des. Many stu­dies ana­lyse sur­vey data in which respond­ents ans­wer ques­ti­ons eit­her online, over the phone, or in per­son. The pri­mary inte­rest lies in cap­tu­ring as many indi­vi­du­als as pos­si­ble to apply quan­ti­ta­tive methods. Howe­ver, two par­ti­cu­lar pro­blems have emer­ged. Firstly, peo­ple who are poli­ti­cally enga­ged are much more likely to par­ti­ci­pate in sur­veys than those who are less poli­ti­cally inte­gra­ted („overs­am­pling“). Secondly, actual beha­viour does not always align with respond­ents’ repor­ted ans­wers. This often occurs due to per­cei­ved social expec­ta­ti­ons (Phil­ipps and Clancy, 1972), wher­eby voting is seen as soci­ally accep­ta­ble and not voting is soci­ally sanc­tioned. Respond­ents tend to claim they voted even if they did not, which distorts the results („over­re­port­ing“ of voter turn­out). Both pro­blems have been docu­men­ted in various social and natio­nal con­texts (Selb and Mun­zert 2013; Scia­rini and Gold­berg 2016) and can be miti­ga­ted, at least in count­ries with acces­si­ble voter regis­ters, through mea­su­res such as sta­tis­ti­cal weight­ing and com­pa­ring repor­ted and actual behaviour.

Howe­ver, even if respond­ents par­ti­ci­pate in the sur­vey but refuse to ans­wer cer­tain ques­ti­ons („item non-response“), a pro­ble­ma­tic bias can emerge in the data. In this case, the goal of coll­ec­ting com­pa­ra­ble infor­ma­tion from all respond­ents and ana­ly­sing it is not achie­ved. Alva­rez and Li (2022) inves­ti­gate whe­ther it is pos­si­ble to iden­tify respond­ents who refuse to ans­wer out of bore­dom or inat­ten­ti­ve­ness in online sur­veys. Ber­in­sky (2017) argues for appro­priate ques­tion for­mu­la­tion to avoid over­whel­ming respond­ents. He makes a sur­pri­sing obser­va­tion for many rese­ar­chers and/or poli­ti­cally inte­res­ted indi­vi­du­als: „most of the peo­ple, most of the time, do not pay atten­tion to poli­tics“ (p. 317).

In recent years, rese­arch has incre­asingly focu­sed on the phe­no­me­non of respond­ents con­sciously with­hol­ding or pro­vi­ding false infor­ma­tion about their sym­pa­thies towards, for exam­ple, right-wing popu­list par­ties. These indi­vi­du­als may feel that their poli­ti­cal opi­ni­ons are soci­ally unde­si­ra­ble or they may reject par­ti­ci­pa­tion in sci­en­ti­fic sur­veys a priori, per­cei­ving sci­en­ti­fic stu­dies – per­haps even in line with their own poli­ti­cal beliefs – as untrust­wor­thy. When there is a sys­te­ma­tic rela­ti­onship bet­ween indi­vi­du­als’ cha­rac­te­ristics or atti­tu­des and non-response, it results in a pro­ble­ma­tic dis­tor­tion of the results. The goal of ana­ly­sing such data is to iden­tify pre­cis­ely these sys­te­ma­tic pat­terns, and if entire groups refuse to par­ti­ci­pate in the sur­vey, it beco­mes impos­si­ble to do so. Howe­ver, there might also be no con­nec­tion to their spe­ci­fic poli­ti­cal ori­en­ta­tion, and the lack of response may sim­ply be arbitrary.

Series on Inequality and Power

Gro­wing ine­qua­lity is one of the most signi­fi­cant pro­blems of our time. At the same time, sci­en­ti­fic inte­rest is incre­asing and pro­vi­ding new insights with a view to the most pres­sing ques­ti­ons and ans­wers on various dimen­si­ons of ine­qua­lity and its under­ly­ing power structures.

For the debate series „Ine­qua­lity and Power“, doc­to­ral stu­dents from the doc­to­ral pro­gramme „Poli­ti­cal Eco­nomy of Ine­qua­lity“ at the Insti­tute for Socio-Eco­no­mics at the Uni­ver­sity of Duis­burg-Essen have writ­ten down these new fin­dings. In the artic­les, the doc­to­ral stu­dents, who are fun­ded by the Hans Böck­ler Foun­da­tion, pre­sent par­tial results of their rese­arch and dis­cuss rela­ted socie­tal chal­lenges as well as poli­ti­cal pathways. With a focus on ine­qua­lity dimen­si­ons and under­ly­ing power rela­ti­ons, the the­ma­tic arc ran­ges from poverty and taxa­tion to labour mar­ket, gen­der equa­lity or cli­mate policy. Through the the­ma­tic breadth and diver­sity of the methods used, the aut­hors seek to initiate a wider socie­tal debate on how to coun­ter rising inequality.

The series has been published in Ger­man lan­guage in the Makro­nom maga­zine bet­ween April and June 2023. We also docu­ment the series here at ifs­ob­log.

Who Provides „No Response“?

In lon­gi­tu­di­nal stu­dies like the Ger­man Socio-Eco­no­mic Panel (SOEP), which has been con­duc­ting com­pre­hen­sive sur­veys with house­holds across Ger­many since 1984, par­ti­ci­pants can choose to pro­vide „no response“ to all ques­ti­ons (data set: SOEP-Core, v37, EU Edi­tion). In the fol­lo­wing, we examine the com­po­si­tion of those who chose „no response“ when asked about their voting decis­ion in the 2017 fede­ral elec­tion in the SOEP. The respond­ents had various opti­ons for indi­ca­ting how they dis­tri­bu­ted their votes among the par­ti­ci­pa­ting par­ties. In 2017, only 7.7% of the respond­ents did not pro­vide an ans­wer, making it dif­fi­cult to derive sta­tis­ti­cally signi­fi­cant cor­re­la­ti­ons. Cramér’s V mea­sure showed no sta­tis­ti­cal signi­fi­cance for the obser­ved irre­gu­la­ri­ties in the cal­cu­la­tion. Howe­ver, exami­ning the com­po­si­tion of the group still yields inte­res­t­ing insights.

For both years, respond­ents could indi­cate whe­ther they did not vote (if eli­gi­ble), which party or par­ties they voted for, or whe­ther they wan­ted to pro­vide „no response.“ Inva­lid votes or inva­lid respon­ses within the sur­vey were excluded from this analysis.

In the 2017 fede­ral elec­tion, the group that chose „no response“ con­sis­ted of 1,859 indi­vi­du­als, while 19,853 respond­ents indi­ca­ted their voting decis­ion. This means that the over­all ratio was 92.3% respon­ded, 7.7% did not. I was par­ti­cu­larly inte­res­ted in how these num­bers dif­fe­red when we look at gen­der, edu­ca­tion level, inte­rest in poli­tics, poverty sta­tus, and income. The fol­lo­wing ana­ly­sis takes into account the size of the groups, i.e., whe­ther, for exam­ple, more men or women were sur­veyed, as it pro­por­tio­nally repres­ents the response behaviour.

Women (gen­der is mea­su­red as binary in the SOEP) more often chose „no response“ (8.3%) than men (7.1%). Respond­ents with a secon­dary school degree (Real­schul­ab­schluss), con­side­red as „inter­me­diate edu­ca­tion level,“ par­ti­cu­larly fre­quently refu­sed to pro­vide an ans­wer. Inte­res­t­ingly, respond­ents with a lower secon­dary school degree (Haupt­schul­ab­schluss) or no degree dis­c­lo­sed their voting par­ti­ci­pa­tion at the same rate as the highly edu­ca­ted group.

6.9% of those living below the poverty line refu­sed to ans­wer, while the pro­por­tion was 7.8% among those who were not poor. This result shows that the eco­no­mic­ally dis­ad­van­ta­ged in this ana­ly­sis were more likely to pro­vide ans­wers com­pared to those with house­hold inco­mes (weigh­ted) above the poverty line.

The hig­her the level of poli­ti­cal inte­rest, the hig­her the pro­por­tion of respond­ents who ans­we­red. While „no“ or „not much“ inte­rest leads to refu­sal rates of 9.4% and 9.1%, respec­tively, it decrea­ses to 6.1% for strong inte­rest and 4.8% for very strong inte­rest – the lowest refu­sal rate obser­ved here. Thus, poli­ti­cal inte­rest appears to play a signi­fi­cant role in the decis­ion of whe­ther respond­ents pro­vide their voting decision.

To repre­sent the income dis­tri­bu­tion of all respond­ents and divi­ded into pro­vi­ded or with­held respon­ses, I first cal­cu­la­ted income deci­les. This divi­des the group of respond­ents into ten equal-sized groups. The lowest 10% repres­ents those with the lowest income, the second decile includes respond­ents who have bet­ween 10% and 20% of the income dis­tri­bu­tion, and so on. This makes the data more robust against out­liers due to par­ti­cu­larly low or high inco­mes. The fol­lo­wing gra­phic shows how inco­mes are dis­tri­bu­ted within each group proportionally.

The gra­phic shows the „ker­nel den­sity esti­ma­tors,“ i.e., the pro­ba­bi­lity that a respon­dent has the dis­played income if they belong to the respec­tive group. Indi­vi­du­als who pro­vi­ded „no response“ (yel­low line) have a hig­her pro­ba­bi­lity of having inco­mes in the lower range of up to 1,500 euros per month.

The median income for the entire group of respond­ents is 1,666.67 euros, for those who pro­vi­ded respon­ses it is 1,736.67 euros, and for those who with­held respon­ses, it is 1,700 euros.

Politically interested individuals have the lowest dropout rate

When ana­ly­sing voting beha­viour using data from the SOEP, it beco­mes appa­rent that we obtain respon­ses pri­ma­rily from respond­ents who have a hig­her inte­rest in poli­tics, which rai­ses con­cerns about data coll­ec­tion. This con­tri­bu­tion by no means ques­ti­ons the qua­lity of the SOEP, whose staff ensu­res a valuable data source for Ger­many through con­stant qua­lity con­trol and ela­bo­rate weight­ing pro­ce­du­res. Rather, it aims to ques­tion whose opi­ni­ons we are actually cap­tu­ring with ques­ti­on­n­aires and whose opi­ni­ons are „fal­ling by the wayside.“

One’s own poli­ti­cal beha­viour is clo­sely lin­ked to emo­ti­ons and lear­ned beha­viours (e.g., Marx 2019). So why do peo­ple refuse to pro­vide insight into their voting beha­viour? Is it sim­ply a lack of inte­rest in the ques­tion, a lack of recoll­ec­tion, or dis­trust towards the inter­viewer? Or are there enti­rely dif­fe­rent reasons?

These ques­ti­ons could be spe­ci­fi­cally exami­ned in per­so­nal inter­views or focus groups to achieve an over­all impro­ve­ment in data coll­ec­tion. The appeal of the exten­sive and ela­bo­rate data coll­ec­tion in the SOEP lies pre­cis­ely in the com­pa­ra­bi­lity of data across various con­texts. The pre­re­qui­site for this is the uni­form cap­ture of the tar­get popu­la­tion, i.e., the group about which rese­ar­chers aim to make state­ments. In the case of the SOEP (and other large panel data­sets), this tar­get popu­la­tion is none other than the entire popu­la­tion of Ger­many. Howe­ver, when cer­tain groups fail to ans­wer the ques­ti­ons for unknown reasons, it is cru­cial that we address this phe­no­me­non of „non-response“ to spe­ci­fic ques­ti­ons more closely.

The Political Economy of Inequality

The PhD pro­gram „Poli­ti­cal Eco­nomy of Ine­qua­lity” inves­ti­ga­tes ext­ent, cau­ses, and con­se­quen­ces of rising socio-eco­no­mic ine­qua­lity. While eco­no­mic aspects of ine­qua­lity are a key focus of the rese­arch, they are always con­tex­tua­li­zed with non-eco­no­mic dimen­si­ons of ine­qua­lity. Poli­ti­cal, social, and eco­lo­gi­cal cau­ses and con­se­quen­ces of mate­rial ine­qua­lity are sys­te­ma­ti­cally inte­gra­ted into the ana­ly­sis. The rese­arch at the Insti­tute for Socio-Eco­no­mics is cha­rac­te­ri­zed by an inter­di­sci­pli­nary and applied socio-eco­no­mic approach. See all blog posts by mem­bers of the pro­gram.

This article has first been published at makronom.de.

Brief summary

There can be various metho­do­lo­gi­cal pro­blems with sur­vey data on voting beha­viour, such as the overs­am­pling of e.g. poli­ti­cally inte­res­ted respond­ents, false accounts of respond­ents or item non-response. The ana­ly­sis of item non-response in Ger­man panel data con­firms that respond­ents with lower poli­ti­cal inte­rest and lower income are more likely to not pro­vide an ans­wer on who they voted for in the last natio­nal elec­tions. This indi­ca­tes that infor­ma­tion for the more afflu­ent and poli­ti­cally inte­res­ted citi­zens is more abun­dant for rese­arch and public dis­cus­sion, pos­si­bly ske­wing the debate. This adds to the dis­cus­sion about the appro­pria­ten­ess of sur­vey data for rese­arch on poli­ti­cal pre­fe­ren­ces. One pos­si­ble impro­ve­ment would be to word ques­ti­ons more carefully, while com­ple­men­tary methods like per­so­nal inter­views or focus groups might also be requi­red to cir­cum­vent these limitations.