UCB How the Digital Age Affects Modern Employment Paper

International Journal of AdvertisingThe Review of Marketing Communications
ISSN: 0265-0487 (Print) 1759-3948 (Online) Journal homepage: https://www.tandfonline.com/loi/rina20
User generated content presenting brands on
social media increases young adults’ purchase
Mira Mayrhofer, Jörg Matthes, Sabine Einwiller & Brigitte Naderer
To cite this article: Mira Mayrhofer, Jörg Matthes, Sabine Einwiller & Brigitte Naderer (2020) User
generated content presenting brands on social media increases young adults’ purchase intention,
International Journal of Advertising, 39:1, 166-186, DOI: 10.1080/02650487.2019.1596447
To link to this article: https://doi.org/10.1080/02650487.2019.1596447
© 2019 The Author(s). Published by Informa
UK Limited, trading as Taylor & Francis
Published online: 28 Aug 2019.
Submit your article to this journal
Article views: 3964
View related articles
View Crossmark data
Citing articles: 1 View citing articles
Full Terms & Conditions of access and use can be found at
2020, VOL. 39, NO. 1, 166–186
User generated content presenting brands on social
media increases young adults’ purchase intention
€rg Matthesa, Sabine Einwillerb and Brigitte Naderera
Mira Mayrhofera, Jo
Advertising and Media Effects Research Group, Department of Communication, University of Vienna,
Vienna, Austria; bCorporate Communication Research Group, Department of Communication,
University of Vienna, Vienna, Austria
On Facebook, companies not only actively spread branded content
themselves, they also encourage users to do so. Hence, persuasive
messages blend into the stream of content, making it increasingly
difficult for users to identify and cope with this covert advertising
content. In an experimental study, we confronted users to disclosed
advertisements; brand; and user-generated posts allowing us to
discern effects on persuasion knowledge, affective reaction and, in
turn, purchase intention. Furthermore, we manipulated viewer’s
attention to the posts. In line with the Persuasion Knowledge
Model, we found that user-generated content did not trigger
persuasion knowledge and a subsequent negative affect. Thus,
user-generated content led to higher purchase intention compared
to disclosed advertisement and brand posts. Surprisingly,
participants’ heightened attention decreased their negative
affective reaction towards the advertisement post compared to the
brand post. We conclude that policy makers should consider
employing advertising disclosures for user-generated content.
Received 28 June 2018
Accepted 13 March 2019
Social media; covert
advertising; persuasion
knowledge; disclosure
In 2017, Facebook had 2 billion daily users worldwide (Socialbakers 2017). Given its
massive number of users, Facebook leads a pack of social media sites in marketing
spending, which from 2009 to 2016 increased by a staggering 234%. Today, a whopping 72.5% of companies in the U.S. use Facebook for advertising purposes (Moorman
2016). With a vast majority of social media marketers being convinced that Facebook
‘delivers the best ROI among the social networks’ (Newberry 2018), it is an important
and established marketing channel (Choi 2011).
On Facebook’s newsfeed, users are confronted with branded content in three different ways. First, paid advertisement posts can be placed by companies in the target
group’s news feed, in which case brands appear as the sources of the posts. Such
posts are comparable to traditional advertising insofar as companies invest financial
resources into exposing a broad target base of consumers to their persuasive
CONTACT Brigitte Naderer
Advertising and Media Effects Research Group,
Department of Communication, University of Vienna, W€ahringerstr. 29, 1090 Vienna, Austria.
! 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives
License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in
any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
messages. To comply with Facebook’s policy for advertisers, such posts have to be
marked as advertisements by a disclosure. In 2018, five million businesses actively
placed paid advertisement campaigns on Facebook (Newberry 2018). Second, brands
can create a brand page to solicit a brand community, i.e. users who ‘like’ the brand
page. Over 70 million businesses operate their own Facebook page, thus taking advantage of this promotion opportunity (Newberry 2018). In those instances, however, only
users who have already ‘liked’ the brand on Facebook are exposed to the companies’
posted content. Third, users can post content that contains brand references to their
Facebook pages and, appearing as its sole source, thereby exposing their entire
Facebook network to the branded content (Facebook 2017).
The latter option, user-generated brand content, is a highly discussed marketing
tool. Especially business media, such as Forbes (Olenski 2017) or Adweek (Merckel
2017) praise user-generated content as highly advantageous covert marketing tool for
companies, as it blends into the editorial social media content. Regulators might be,
however, rightfully concerned that users are no longer able to identify persuasive content on social media. This is crucial though, as we know based on the Persuasion
Knowledge Model (PKM) by Friestad and Wright (1994) that in the case of traditional
media, only if viewers identify a certain content as commercial, coping mechanisms
are triggered, which may lead to more critical processing of the message. Prior
research in the online realm found similar results for blog posts (e.g. van Reijmersdal
et al. 2016), vlog posts (De Jans, Cauberghe, and Hudders 2018), native advertising in
articles (Campbell and Evans 2018), or Instagram posts (e.g. Evans et al. 2017). The U.S
Federal Trade Commission advises social media users to employ media literacy techniques to identify branded content. Thus, users should reflect on questions like ‘who
created or paid for the ad, and why?’ (FTC 2013). However, answering those questions
might not be so easy anymore in times of social media marketing, when the explicitly
stated source of a posting can be a user and FTC guidelines of disclosing advertised
content are bluntly ignored by businesses (Fletcher 2017). Hence, it is crucial to generate insights on how the processes proclaimed by the PKM manifest for the different
forms of branded posts on social media.
Furthermore, even if branded posts are disclosed as advertisement, studies on
source and disclosure effects suggest that viewers have a hard time identifying persuasive content because they do not pay enough attention to corresponding indicators (Wojdynski and Evans 2016). In fact, the European Commission (2018) stated that
increasing viewers’ visual attention to advertising disclosures can serve as a remedy
against disguised advertising practices. This could be the case, because heightened
attention can enhance persuasion knowledge (Boerman, van Reijmersdal, and Neijens
2015). Two highly related constructs in this regard are visual attention and cognitive
involvement (Pieters and Wedel 2007). That is, when humans concentrate on a content
(i.e. are highly cognitive involved), their visual attention to the different aspects of this
content is also high. In light of this, one would theorize that higher attention helps
users to understand that they are confronted with persuasive messages.
Yet from literature on information processing, we know that humans only have limited cognitive processing capacities (Lang 2000). Studies indicate that social media use
has led to an extensive increase in the amount of information a user is exposed to,
greatly increasing the cognitive load (Gomez-Rodriguez, Gummadi, and Schoelkopf
2014). Consequently, highly attentive users might not have enough capacity to experience an affective reaction towards a content (Matthes, Schemer, and Wirth 2007;
Janssen et al. 2016). Thus, attention may foster persuasion knowledge, but not necessarily negative affect.
With this study, we are the first to investigate the effects of attention on persuasion
knowledge and affective reaction in the social media environment. In addition, this is
also the first comprehensive experimental study testing the effects of three different
types of branded Facebook posts, i.e. disclosed advertisement posts, brand posts, and usergenerated posts. This allows us to assess viewers’ persuasion knowledge and affective
reaction towards these three different types of posts and subsequent brand outcomes.
The power of user-generated content
Media outlets have often highlighted the value of user-generated branded content.
Forbes has recommended that companies ‘take proactive steps to stimulate the creation of user-generated content’ (Olenski 2017), and the subtitle of an article in the
advertising online journal Adweek has argued that ‘Not only is UGC [user-generated
content] much cheaper to implement, but it is also much more effective’ (Merckel
2017). Citing metrics such as hashtag usage and retweets, the latter publication especially underscored the power of user-generated brand images (Merckel 2017). In the
MIT Sloan Management Review, businesses even get advised to redefine their social
media marketing goals in a way that includes brand engagement, i.e. customers posting branded content, as an ROI indicator (Hoffman and Fodor 2010). Consequently,
numerous studies have explored under which circumstances users are willing to post
branded content. Researchers who have investigated the reasons why users contribute
and create brand-related content have identified personal identity, integration, and
social interaction as major motivations. Many users upload pictures displaying brands
in order to express their connection to a brand’s image and popularity as well as their
inclusion in the social group that uses the brand (Muntinga, Moorman, and Smit
2011). Common among such posts are so-called ‘brand-selfies’, which Sung, Kim, and
Choi (2018, 10) have dubbed an ‘effective means of self-expression’. In a content analysis of posts connected to two apparel brands, Smith, Fischer, and Yongjian (2012)
found that personal identity, integration, and social interaction motivated a third of
user-generated Facebook posts.
More importantly, user-generated brand posts result from marketing efforts such as
real-world tie-ins or contests. In their content analysis of several social media networks,
Ashley and Tuten (2015, 22) found that ‘26/28 brands invited users to share content’.
One industry that often uses covert marketing techniques is the alcohol industry. Realworld tie-ins are the most important content generators on alcohol brand community
pages. Alcohol brands for instance launch branded event series (Nicholls 2012), sponsor sport events (Pinsky et al. 2017), and corresponding hashtags. Another technique
of the alcohol industry, described by Lobstein et al. (2016), are photo- or video-competitions. People are therefore encouraged through these events and competitions to
take pictures and post them on their own social media channels: ‘The personal brand
experience of the fan leads to the circulation of branded information within the
broader mediation of their everyday life within their circle of friends’ (Brodmerkel and
Carah 2013, 279). Carah and Shaul (2016) analyzed two highly successful Smirnoff campaigns that mobilized user-generated branded content on Instagram to promote the
Smirnoff brand. Regarding how user-generated content connected the brand to the
users, they wrote that users’ ‘use of hashtags places the brand within a wider flow of
images related to their own bodies and identities’ (Carah and Shaul 2016, 74).
However, even with the reasons for users to create and share brand-related content
established, it remains unclear whether users exposed to such content will react similarly to how they react to brand-related content posted by commercial sources. In the
area of effect studies, research on the increasingly recognized value of user-generated
brand content is scarce. Hence, we sought to assess how the source of a branded
message affects viewers’ evaluations of the brand.
Several scholars have investigated the effects of having users as the source of persuasive messages. Central to their studies was, however, the likeability, credibility, and
perceived quality of user-generated advertisements (Ertimur and Gilly 2012; Steyn
et al. 2011; Thompson and Malaviya 2013). For instance, Ertimur and Gilly (2012, 126)
found that when viewers were aware that a user-generated post was commercial content, they did not cope with the persuasive intent of the post. Instead, they took the
role of an ad critic and, for example, gave suggestions about the lighting or editing of
the presented pictures.
Other authors have examined the role of users as the source of advertisements,
when testing advertisement disclosures (e.g. De Jans, Cauberghe, and Hudders 2018;
Hwang and Jeong 2016; Kim and Song 2017). Their results do not tackle, however
how different sources affect the activation of persuasion knowledge, because they
depend on the effectiveness of the disclosure manipulation itself – that is, whether
the disclosure made participants realize that the source of the content was a company
and not the user (Kim and Song 2017) and whether that realization triggered negative
affect (Matthes and Naderer 2016). For example, Kim and Song (2017) compared user
generated brand posts with and without disclosure. This however cannot clearly establish the effect of a user as the source of branded content in comparison to brand messages by the company itself.
At the same time, although other scholars have focused on the effects of reviews
or product-related stories shared by users, such forms of advertising are hardly comparable to user-generated brand posts on Facebook, which do not always have a persuasive or informative intent (Smith, Fischer, and Yongjian 2012). This brings us to the
possibly biggest advantage for companies in employing user-generated brand posts
as marketing effort and an area strongly lacking research: the alleged incapability of
social media users to identify it as persuasive content and in turn to cope with it.
Coping with brand messages
A model analyzing the different factors influencing consumers’ understanding of persuasion tactics and their knowledge of how to cope with persuasive attempts is the
PKM (Friestad and Wright 1994, 1995). Persuasion knowledge develops gradually, due
to consumers’ regular exposure to persuasive content. Persuasion knowledge is based
on knowledge about the topic, understanding the persuasive process, and knowing
the agents respectively communicators. In this study, we specifically focus on the latter, i.e. agent knowledge. This aspect consists of the consumer being aware that the
source of a message has a commercial background and, hence, a persuasive intent.
This knowledge should lead users to identify a message as a persuasive attempt and
in turn trigger coping mechanisms. The agent ‘represents whomever a target identifies
as being responsible for designing and constructing a persuasion attempt’ (Friestad
and Wright 1994, 2). In the case of Facebook posts, the source of the posting is always
stated above the content, i.e. the picture or statement. In other words, the PKM suggests that knowing that a brand or company is behind a communication effort will
trigger coping mechanisms aimed at potentially blocking a persuasive intent (Friestad
and Wright 1995).
On Facebook, however, there are two types of posts that present a brand as the
source. Brands can pay to appear in users’ newsfeeds that fit their target group. In
this case, a short disclosure alerts the user to the fact that they are confronted with
paid content. If a user or one of his or her ‘friends’ however, follows a brand page on
Facebook, they agree to be confronted with branded posts and as such no ad-disclosure is presented. Embedded in a stream of miscellaneous content, both types have, at
first glance, only one main difference: The disclosure of the advertising message.
Consequently, it is questionable if users correctly identify persuasive content presented
on social media at all or only in the case of paid posts that contain an ad-disclosure
(Evans et al. 2017).
Past research indicates that the appearance of a disclosure can alert users’ attention
to the fact that they are confronted with a persuasive message, thus trigger users’ persuasion knowledge. This has been indicated for product placement disclosures on TV
(e.g. Boerman, van Reijmersdal, and Neijens 2012, 2015; Boerman and van Reijmersdal
2016; Matthes and Naderer 2016; Tessitore and Geuens 2013), disclosures in advergames (Evans and Hoy 2016; van Reijmersdal et al. 2015), and social media outlets
such as blogs (van Reijmersdal et al. 2016) or Instagram (Evans et al. 2017). Results
hitherto remain inconclusive, however, mainly due to the differences in disclosure
manipulations (see Evans et al. 2017). In other words, whether the disclosure factually
made participants understand that the underlying source of the content is a company
and not a user is of key importance. Some studies, however, have put the effectiveness of certain ad disclosures in covert marketing into doubt (e.g. Wojdynski and
Evans 2016). Following the logic of the PKM, we therefore hypothesize:
H1: Persuasion knowledge is activated a) the least when participants are exposed to usergenerated posts and b) the most when exposed to disclosed advertising posts. Hence, we
expect brand posts to evoke more persuasion knowledge than user-generated posts and
less than disclosed advertising posts.
Activated persuasion knowledge may in turn encourage users to critically contest
the content itself, the source of the content, and the persuasive tactics of the content
(Fransen, Smit, and Verlegh 2015). In a qualitative study, Zuwerink and Cameron
(2003) assessed the variety of coping mechanisms that can be triggered by persuasion
knowledge. Cognitive resistance, such as attitude bolstering, which is the reassurance
of one’s belief, and affective resistance, such as negative affect against the persuasive
attempt, were among the most frequent resistance strategies. In the context of sponsored content, extant research (van Reijmersdal et al. 2016) suggests that affective
resistance co-occurs with cognitive resistance for two reasons. First, research showed
that content which is perceived as advertising often directly triggers negative attitudes, annoyance, and avoidance behaviour (e.g. Mittal 1994; Moriarty and Everett
1994). Second, Friestad and Wright (1994) pointed out that identifying a communication as persuasive intent is a change of meaning. Thus, the persuasive attempt is considered an intruder into the communication context. In other words, if a post is
identified as a persuasive intent while someone is browsing through entertaining and
informative content on their Facebook wall, it might intrude in the Facebook browsing
experience and thereby trigger resistance. The meaning of the persuasive message is
scrutinized, which in turn leads to negative effects on brand evaluations (Evans et al.
€derlund 2015). This was
2017; Hwang and Jeong 2016; Liljander, Gummerus, and So
supported by research on advertisement intrusiveness which has linked the intrusiveness to viewers being annoyed by advertisements (e.g. Ha 1996; Truong and Simmons
2010). Consequently, we pose the hypothesis:
H2 : The higher a participant’s level of persuasion knowledge regarding a Facebook post
the stronger is his/her negative affective reaction against the Facebook post.
Last but not least, the goal of a marketing communication is to influence marketing
outcomes, such as, for example, purchase intent. Extant research has demonstrated in
diverse channels how increased persuasion knowledge diminished marketing outcomes (Boerman, van Reijmersdal, and Neijens 2012; Matthes, Schemer, and Wirth
2007). Interestingly, Wei, Fischer, and Main (2008) have found the effect to especially
hold for unfamiliar brands. We predict that this appears on account of the explained
triggered coping mechanism of negative affect (Zuwerink and Cameron 2003).
Therefore, we suggest that:
H3 : The stronger participants’ negative affect against a Facebook post, the stronger the
decrease in their purchase intention of the brand presented within the post.
The role of attention in the persuasive process
In extant research, different aspects of attention have been studied (Park and McClung
1986; Wojdynski and Evans 2016). Two concepts that researchers have connected are
visual attention and cognitive involvement (Pieters and Wedel 2007). For instance,
involvement scales measure aspects as ‘I concentrated on the story’ (Matthes,
Schemer, and Wirth 2007, 499) and cognitive involvement manipulations demand participants to ‘pay specific attention’ to the depicted content (Park and McClung 1986,
546). This is highly connected with participants looking more attentively at all aspects
of a content (i.e. visual attention, Pieters and Wedel 2007).
Why is the degree of attention important? Especially in the social media environment, attention to posts may be crucial for the persuasion process, as commercial and
non-commercial content are strongly interconnected. In fact, several studies on the
topic of source and disclosure effects have demonstrated that viewers are unaware
that a content has a commercial source, even if the source itself is disclosed or an
advertising disclosure is included in the content (e.g. Boerman, van Reijmersdal, and
Neijens 2012, 2015; Nelson and Park 2015; Kim, Pasadeos, and Barban 2001; Wojdynski
and Evans 2016). Eye-tracking studies have established that one issue is a lack in visual
attention of the viewers. If users don’t pay attention to the posts, they may not process the information provided in, for instance, disclosures. As a consequence, the
intended effect of disclosures on persuasion knowledge can hardly be achieved (see
Wogalter and Laughery 1996 for results on warning signs and labels in general, see
Boerman, van Reimersdal, and Neijens 2012 for a study on product placement disclosures). For instance, Wojdynski and Evans (2016) found that in the case of a news
story, viewers did not visually engage with an advertising disclosure positioned above
the story. This position of the disclosure, however, matters for Facebook posts, as the
source of a message is positioned above the content. Hence, when viewers are confronted with advertising or brand posts, a lack of attention to the posts makes it
unlikely that sources and disclosures are processed. This, in turn, may impede the generation of persuasion knowledge. Therefore we hypothesize:
H4: In the disclosed advertising and brand post conditions, opposed to the usergenerated content condition, users in the high-attention condition will have a higher
persuasion knowledge, compared to users in the normal-attention condition.
Regarding negative affect, the role of attention, however, is less clear. As already
mentioned, cognitive resources of viewers are typically limited (Nairn and Fine 2008).
That is, an information overload could occur when the amount of input to a system
exceeds its processing capacity (Janssen et al. 2016). Yet negative affect towards an
advertising attempt requires a substantial amount of cognitive resources. Following
this assumption, users that process posts with high attention have only limited resources left to experience an affective reaction towards messages (Nairn and Fine 2008). In
fact, this mechanism was found for traditional television programming (e.g. Park and
McClung 1986), as well as for integrated content, such as product placements (e.g.
Janssen et al. 2016; Matthes, Schemer, and Wirth 2007). For instance, Matthes,
Schemer, and Wirth (2007) observed that levels of involvement exerted an effect on
negative affective outcomes: Highly involved viewers with low levels of persuasion
knowledge were least critical towards the embedded brands. Along the same lines,
Janssen et al.’s (2016) results indicate that depletion can impact the critical processing
of information. The authors did find that depleted participants were less critical
towards the embedded brands. Again, these findings can be explained on the theoretical basis of the limited capacity model which suggests that humans have only a limited capacity of critically processing the abundance of information provided (Lang
2000). Especially in the social media environment, heightened attention to posts might
lead to a cognitive overload for users, as they are confronted with a variety of contents in those posts. This cognitive overload could impede their experience of a negative affect towards content. In other words, in the case of processing social media
content, attention to the posts as a whole may require substantial cognitive resources.
These resources, however, are necessary to raise negative affect in response to advertising or brand posts. That is, even though viewers may realize the persuasive attempt,
they may lack the necessary cognitive resources to generate negative affect. However,
to our knowledge, there is no research yet on the effects of attention on negative
affect in regard to social media content. We therefore refrain from formulating an
hypothesis, and ask:
RQ1: How will attention influence participant’s affective reaction regarding the (a)
disclosed advertising post, (b) brand post and (c) user-generated post?
To conclude, there is no research yet comparing the effects of user-generated
posts, brand post, and disclosed advertising posts as sources of branded Facebook
posts. We expect that users as sources of branded Facebook posts will trigger less persuasion knowledge regarding the post compared to brand posts as well as disclosed
advertising posts, therefore impeding negative affect. Hence, user-generated brand
posts should more positively affect purchase intention than disclosed advertising posts
or brand posts. Furthermore, we test the role that attention could play in the persuasion process. For the full effect model see Figure 1.
The experiment had a 3 (source: disclosed advertisement versus brand post versus
user-generated post) ! 2 (attention: high versus normal) between-subject design.
Analyses were conducted with a total of N ¼ 293 college students above the legal
drinking age (Mage ¼ 22.57; SD ¼ 3.58; 78.2% female) in the university laboratory of the
Department of Communication at the University of Vienna in May 2017. We followed
the APA ethical guidelines; hence all participants signed an informed consent form
before entering our university laboratory, where the study was conducted. Then, we
randomly assigned them to an attention condition consisting of a textual instruction.
We operationalized attention via two aspects, cognitive involvement adapted from
Park and McClung (1986) and visual attention. In the high-attention condition, we
asked the participants to concentrate on all aspects of the posts, to memorize them
and to look carefully at all aspects of them. In the normal-attention condition, we
encouraged their usual browsing behaviour (see Appendix). In a second step, we randomly assigned participants to one of the source conditions. Participants went
through the 20 Facebook posts in randomized order, fifteen identical nonrelated posts
and five source manipulated alcohol posts. After the study, they were debriefed about
the topic of the study and the risks of excessive alcohol consumption.
All participants saw fifteen identical, nonrelated posts aimed at embedding the five
manipulated posts in a realistic setting. Furthermore, participants saw five posts featuring the ‘Ciroc’ brand. Each post presented a different photo related to the ‘Ciroc’
brand. The ‘Ciroc’ posts were identical, except for the source being a disclosed advertising by the brand ‘Ciroc’, a brand post by the same brand, or a user-generated brand
post featuring the brand ‘Ciroc’. In other words, the remaining five posts were identical except for the source of the posts and, respectively, the included ad disclosure
Figure 1. Model of all hypothesized paths and research questions.
(Evans et al. 2017). Hence, each participants either saw all five different ‘Ciroc’ posts
by a disclosed advertising, a brand post or a user-generated post, the source being
treated as between-subject factor (Stimulus material is available upon request).
We chose an alcohol brand as the target brand because we deemed it important
that the user-generated content seems realistic to the participants. Regarding the post
engagement rate – which involves response rates to fan posts, counted likes, comments and shares – the alcohol industry is the clear forerunner (Socialbakers 2012).
Brand pages post content up to twice a day (Carah 2014), strongly encouraging users’
engagement with real-world tie-ins; interactive games; sponsored online events and
invitations to drink’ (Nicholls 2012, 487). Hence we assumed that users are used to
this kind of content posted by brand as well as user sources.
We pre-tested the stimulus pictures to assure that their content did not appear more
realistic for either a post placed by a company or a user-generated post. This was of
high importance, in order to exclude effects from the content itself. Student participants (N ¼ 73) saw twenty Facebook posts, among them the manipulated posts. They
were randomly assigned to a condition with the user being the post source (n ¼ 25) or
the brand being the source (n ¼ 48). We showed participants only the content of the
manipulated posts, i.e. a statement and a picture but without any source. We did this
for all manipulated posts one by one. Subsequently, we asked participants, if they felt
that a user or brand would be equally probable to post such content (1 ¼ not at all,
5 ¼ very much). Participants evaluated the probability for picture one (M ¼ 3.74,
SD ¼ 1.11), picture two (M ¼ 3.53, SD ¼ 1.27), picture three (M ¼ 3.44, SD ¼ 1.21), picture
four (M ¼ 3.22, SD ¼ 1.29), and picture five (M ¼ 3.74, SD ¼ 1.33). All content was
deemed to be as probable to be posted by a user as by a brand. Additionally, participants were asked, if they could imagine seeing such a post on Facebook (1 ¼ not at
all, 5 ¼ very much). All pictures were deemed expectable as a Facebook post: picture
one (M ¼ 4.33, SD ¼ 0.77), picture two (M ¼ 3.89, SD ¼ 1.04), picture three (M ¼ 3.78,
SD ¼ 1.24), picture four (M ¼ 4.0, SD ¼ 1.08), and picture five (M ¼ 4.38, SD ¼ 0.72). As
such, the content of the post was deemed appropriate for the study.
We assessed participants’ attention using two statements on a 5-point scale adapted
from Park and McClung’ cognitive involvement manipulation (1986) (‘I tried to memorize the posts’, ‘I concentrated on the posts’) and three statements focusing on participants’ visual attention (‘I looked at all the aspects of the post’, ‘I looked at the posts
attentively’, ‘I took time to look at the posts’). Following Carpenter (2018), we conducted an explorative factor analysis with oblique rotation with the five items, which
indicates that both elements are part of the same concept, which we employed as a
measure of attention. The analyses yielded one factor, explaining 71.71% of the variance. Hence, the statements formed a reliable index (a ¼ 0.90; M ¼ 3.87, SD ¼ 0.95).
Furthermore, we measured source recall as an open question. Comparable to van
Reijmersdal et al. (2016), we measured the participants’ persuasion knowledge using
three items (‘These posts were advertising’, ‘These posts were posted without commercial interest’ (recoded), ‘These posts were posted to advertise a product’) on a 5-point
scale (a ¼ 0.90; M ¼ 4.18, SD ¼ 1.15). Negative affective reaction (van Reijmersdal et al.
2016) to the post was also measured using three items on a 5-point scale (‘I was irritated/annoyed/enraged by those posts’). The formed index was reliable (a ¼ 0.78;
M ¼ 2.30, SD ¼ 1.04). In addition, we assessed participants’ purchase intention for the
embedded brand, Ciroc, using three items (‘I would buy Ciroc Vodka’, ‘I would by
other alcoholic products of the Ciroc brand’, ‘I am interested in where to buy Ciroc
Vodka’) on a 5-point scale (a ¼ 0.78; M ¼ 1.92, SD ¼ 0.93). As a control variable, we
included participants’ alcohol consumption, stated on a single 5-point scale item
(1 ¼ never, 2 ¼ less than once a month, 3 ¼ two to four times a month, 4 ¼ two to three
times a week, 5 ¼ four times a week or more; M ¼ 2.94, SD ¼ 0.85). It was to be expected
that participants’ current level of alcohol consumption might predict their interest in
buying alcohol (e.g. Alhabash et al. 2015). Further, we included age (mean-centred) as
a control variable, as extant studies have shown that age can affect viewers’ perception of alcohol related content (Mayrhofer and Naderer 2019).
Randomization and manipulation checks
A randomization check for gender (v2 ¼ 5.75, df ¼ 5, N ¼ 293, p ¼ .33) was successful.
Furthermore, the manipulation check of attention in the main study was successful.
Hence, in the normal attention condition, users indicated a significantly lower level of
attention (M ¼ 3.50, SD ¼ 0.91) compared to the high attention condition (M ¼ 4.27,
SD ¼ 0.83) (t(291) ¼ #7.55, p < .001). Additionally, we dummy coded the open source recall question with a margin of one letter. Participants in the high attention condition remembered the correct source of the brand posts significantly more than those in the normal attention condition. Namely, 67.1% (n ¼ 94) of high attentive participants remembered the source, against only 42.5% (n ¼ 65) (v2 ¼ 17.91, p < .001). 176 M. MAYRHOFER ET AL. A manipulation check on a separate sample with similar demographics as the main sample was conducted to assure that participants understood the difference between disclosed advertisement and brand posts (N ¼ 36). They saw both types of posts (Post 1 ¼ brand post; Post 2 ¼ advertisement post) and had to agree or disagree with eight statements that were correct or incorrect (e.g. ‘I would only see post 1 if I had already liked the brand’, ‘Every user could be exposed to post 2 in their newsfeed’, ‘Post 2 is disseminated by a brand, without paying Facebook for it’. see Appendix). All statements were judged correctly by at least 80.6% (n ¼ 29). Data analysis We calculated a moderated mediation model to test our hypotheses with 5000 bootstrapping samples. We used model 85 with SPSS Process (Hayes 2018). The experimental conditions were dummy coded. First, the user-generated brand post condition was used as the reference group. Second, we used the disclosed advertising post condition as reference group to assess the relevance of the advertising disclosure. Attention had two codes (0 ¼ normal and 1 ¼ high attention). We computed interaction terms between source and attention. To assure the robustness of the results, we tested all possible direct effects (see Table 1). Results Persuasion knowledge In a first step, we investigated the effects on participants’ persuasion knowledge. We found a main effect of the disclosed advertising post condition (b ¼ 1.13, p < .001, LLCI ¼ .740; ULCI ¼ 1.516) and the brand post condition (b ¼ .92, p < .001, LLCI ¼ .495; ULCI ¼ 1.354) compared to the user-generated post. Thus, if a brand is the source of a message, the user’s persuasion knowledge is increased. In other words, the user as the source induces the least persuasion knowledge. Therefore, Hypotheses 1a was supported. There was, however, no main effect of the attention condition (b ¼ .33, p ¼ .11, LLCI ¼ #.071; ULCI ¼ .737). Also, we could not support H4, as attention did not moderate the effect of the post source on persuasion knowledge (disclosed advertising post $ attention: b ¼ #.09, p ¼ .75, LLCI ¼ #.647; ULCI ¼ .468, brand post $ attention: b ¼ .15, p ¼ .63, LLCI ¼ #.459; ULCI ¼ .761). For all results see Table 1. To establish the effect of the advertising disclosure, we ran the model using the advertising condition as a reference group. Again, focusing on the level of persuasion knowledge, we found a negative main effect of the user-generated brand post condition (b ¼ #1.13, p < .001, LLCI ¼ #1.516; ULCI ¼ #.740) in comparison to the advertising condition but no effect of the brand post condition (b ¼ #.20, p ¼ .33, LLCI ¼ #.610; ULCI ¼ #.740). Thus, the disclosure did not increase the users’ awareness for the persuasive intent. Consequently, Hypothesis 1b was not supported, as the disclosed advertising did not induce significantly more persuasion knowledge than the brand post (for the descriptive results on persuasion knowledge see Table 2). As can be seen in Table 3, no additional main or interaction effects occurred. INTERNATIONAL JOURNAL OF ADVERTISING 177 Table 1. Path analysis: user-generated post inserted as reference group. Persuasion knowledge Variables Disclosed advertising post Brand post Attention Age Alcohol consumption Disclosed advertising post$ attention Brand post$ attention Persuasion knowledge Affective reaction Explained variance Affective reaction SE .20 .22 .21 .02 .07 .28 .31 b 1.13$$$ .92$$$ .33 #.01 .08 #.09 .15 SE .21 .22 .21 .02 .07 .28 .31 .06 b .23 #.02 .14 #.01 #.06 #.45 .21 .13$ .22 Purchase intention .05 b #.11 .04 #.10 #.02 .17$$ .15 #.05 #.04 #.22$$$ .11 SE .18 .20 .18 .01 .06 .25 .27 .05 .05 Note: N ¼ 293. $p < .05. $$p < .01. $$$p < .001. Table 2. Descriptive results: persuasion knowledge by source and attention. Attention Persuasion knowledge User-generated post Brand post Disclosed advertising post High Normal 3.65 (1.26) 4.74 (0.76) 4.71 (0.68) 3.32 (1.17) 4.26 (1.27) 4.47 (0.88) Note: N ¼ 293. Table 3. Path analysis: disclosed advertising post inserted as reference group. Persuasion knowledge Variables User-generated post Brand post Attention Age Alcohol consumption User-generated post$ attention Brand post$ attention Persuasion knowledge Affective reaction Explained variance $p < .05. $$p < .01. $$$p < .001. Affective reaction Purchase intention b SE b SE b SE #1.13$$$ #.20 .24 #.01 .08 .09 .24 .20 .21 .20 .02 .07 .28 .30 #.23 #.25 #.31 #.01 #.06 .45 .66$ .13$ .21 .21 .19 .02 .07 .28 .30 .06 .11 .14 .05 #.02 .17$$ #.15 #.20 #.04 #.22$$$ .18 .18 .17 .02 .06 .25 .27 .05 .05 .22 .05 .11 Affective reaction In a second step, we investigated participants’ affective reaction. We found no main effects of the disclosed advertising post condition (b ¼ .23, p ¼ .26, LLCI ¼ #.175; ULCI ¼ .642) and the brand post condition (b ¼ #.02, p ¼ .94, LLCI ¼ #.458; ULCI ¼ .425). No additional main or interaction effects occurred. However, persuasion knowledge significantly influenced affective reaction (b ¼ .13, p ¼ .03, LLCI ¼ .011; ULCI ¼ .243). Hence, Hypothesis 2 was supported. Attention had no main effect (b ¼ .14, p ¼ .51, LLCI ¼ #.268; ULCI ¼ .542) or interaction effects of the disclosed 178 M. MAYRHOFER ET AL. advertising post(b ¼ #.45, p ¼ .11, LLCI ¼ #1.005; ULCI ¼ .108) and brand post condition (b ¼ .21, p ¼ .49, LLCI ¼ #.395; ULCI ¼ .823) compared to the user-generated post (Table 1). We then ran the analysis with the disclosed advertising post as reference group and uncovered that the interaction effect of the brand post condition compared to the advertising condition and attention (b ¼ .66, p < .03, LLCI ¼ .067; ULCI ¼ 1.257) was significant. This indicates that highly attentive participants showed lower levels of negative affect reactions in the disclosed advertising post, compared to the brand post condition (Table 3). Purchase intention In a last step, we assessed purchase intention as our main outcome measure. We found no main effect of the disclosed advertising post condition (b ¼ #.11, p ¼ .56, LLCI ¼ #.463; ULCI ¼ .250) and the brand post condition (b ¼ .04, p ¼ .85, LLCI ¼ #.348; ULCI ¼ .421). There was no main effect of attention (b ¼ #.10, p ¼ .56, LLCI ¼ #.457; ULCI ¼ .249). Also, the interaction effect of the disclosed advertising post condition and attention (b ¼ .15, p ¼ .54, LLCI ¼ #.334; ULCI ¼ .639) and the interaction effect of the brand post condition and attention (b ¼ #.05, p ¼ .86, LLCI ¼ #.578; ULCI ¼ .483) were not significant. Age (b ¼ #.02, p ¼ .10, LLCI ¼ #.054; ULCI ¼ .005) had no main effect. Persuasion knowledge had no direct effect (b ¼ #.04, p ¼ .50, LLCI ¼ #.137; ULCI ¼ .067). Yet we found alcohol consumption to have a main effect (b ¼ .17, p < .01, LLCI ¼ .044; ULCI ¼ .287) on participants intention to purchase the embedded brand. Thus, the level of users’ alcohol consumption positively increased users’ intention to buy the advertised brand, independent from condition. More importantly, we also found a negative effect of affective reaction towards the posts (b ¼ #.22, p < .001, LLCI ¼ #.323; ULCI ¼ #.120). Consequently, Hypothesis 3 was supported (Table 1). The results for purchase intention stayed constant when putting the disclosed advertising post as reference group (Table 3). We then further examined the mediation path of source on purchase intention via level of persuasion knowledge and via negative affective reaction. The indirect effect of the disclosed advertising post on purchase intention via persuasion knowledge and via negative affective reactions compared to the user generated post was significant (b ¼ #.03; LLCI ¼ #.076; ULCI ¼ #.001). Furthermore, compared to the user-generated content, the brand post lead to a negative indirect effect via persuasion knowledge and via negative affective reaction on purchase intention (b ¼ #.03; LLCI ¼ #.070; ULCI ¼ #.001). Discussion This study is the first to experimentally manipulate and, thus, comprehensively assess the effects of different types of brand posts on users. While keeping the content of the posts constant, we tested how different sources affected the viewers’ perceptions of the posts and their purchase intention for the branded product. Furthermore, we tested the role of attention, based on visual attention, as well as cognitive INTERNATIONAL JOURNAL OF ADVERTISING 179 involvement, on persuasion knowledge and affective reaction for the three different posts. Lastly, we controlled for age and alcohol consumption, which were shown in prior studies to affect viewers’ perception of alcohol-related content (Alhabash et al. 2015; Mayrhofer and Naderer 2017). A crucial insight is that user-generated branded content, which might be considered covert advertisement, significantly decreased persuasion knowledge of the viewers. As such, coping mechanisms were not triggered, thus dampening the negative affect towards the post, which in turn led to a higher purchase intention as compared to both posts by the brand itself (disclosed advertising post and brand posts). In other words, the coping mechanisms that users employ to resist persuasive intent are not triggered if the displayed source of the persuasive content is a fellow user. As expected, this is in line with Friestad and Wright’s (1994) PKM, which posits agent knowledge as a crucial information to trigger persuasion knowledge. Furthermore, it supports van Reijmersdal et al.’s (2016) assumption that users employ negative affect to cope with persuasive messages, which decreases marketing outcomes, such as purchase intention. This finding has important implications for further research as well as for policy makers and consumers. To start with, user-generated brand content is a marketing tool used by a wide array of industries and praised by professionals. In contrast to this tremendous relevance, research on the effects of user-generated content on viewers is scarce. For instance, the idea of the exorbitant profitability of user-generated content as described in Forbes (Fromm 2016) or Adweek (Boachie 2018) is based on individual companies and opinions, instead of research studies. Furthermore, assuming that usergenerated content indeed has benefits compared to traditional advertising, the processes behind this fact remain unclear. We tried to address this topic focusing on the PKM (Friestad and Wright 1994), however, other mechanisms such as a mere-exposure effect (Strick, van Baaren, Holland, and van Knippenberg 2009) should be investigated. While it has to be stated in general that consumers have the right to be informed if they confront a persuasive intent, the lack of such information is especially worrisome when it comes to health damaging or risky products, such as alcohol. While we acknowledge that alcohol related posts are often not directly connected to a brand and posted by users for different reasons (Ridout, Campbell, and Ellis 2012; Westgate and Holliday 2016), it has to be underlined that alcohol companies are actively encouraging users to post such pictures (Carah 2014; Nicholls 2012). As a matter of fact, user-generated content is especially attractive for the alcohol industry, since with this technique pictures and messages presenting the brand can circumvent advertising regulations, such as depicting drinkers below the age of 25, showing excessive drinking and presenting sexual advantages due to alcohol (Lobstein et al. 2016). One option often brought up in the context of making viewers more vigilant is the use of advertising disclosures. This directly leads to the second finding of this study: the ineffectiveness of the advertising disclosure to enhance users’ level of persuasion knowledge. While several studies found that advertising disclosures can trigger persuasion knowledge in blogs (van Reijmersdal et al. 2016) or on Instagram (Evans et al. 2017), this study falls in line with Kim, Pasadeos, and Barban’s (2001) study on advertorials 180 M. MAYRHOFER ET AL. finding no additional effect of a disclosure. As Kim, Pasadeos, and Barban (2001) explained, this might be due to a ceiling effect. In fact, persuasion knowledge regarding the brand post condition was high (M ¼ 4.50, SD ¼ 1.08 measured on a 5-point scale). In other words, the fact that the source of the post was a brand already triggered a considerable level of persuasion knowledge. Interestingly, we did not find any effect of heightened attention on persuasion knowledge, but again this might be due to the mentioned ceiling effect (see Table 2). This points towards the fact that Facebook’s policy on putting disclosures on paid advertisement posts does not tackle the issue of covert marketing. As such, user-generated brand content would also have to be marked as commercial in certain cases, which is, however, challenging to implement since consumers post branded content mostly without commercial intent. Still especially when it comes to alcohol messages, following Brodmerkel and Carah (2013), we suggest policy makers should ‘hold alcohol brands responsible not just for what they say but also for the kinds of audience participation and mediation they invite and encourage’ (2013, 278). For instance, a new definition of what is considered a commercial alcohol brand picture on Facebook should be considered to encompass user-generated material that is created in a direct response to a brand. Pictures using hashtags or tags created for real-world tie-in events, such as the highly successful Smirnoff campaigns (Carah and Shaul 2016; Nicholls 2012) could be marked with a disclosure as well, independently of the pictures’ sources. For instance, if a user publishes a branded post in response to the promise of a free product, this could be understood as a commercial act following recent European jurisdiction. This is the case, as recent court rulings have seen free products as a form of compensation. In other words, if a user attending an event gets a free drink for a branded picture containing a corresponding hashtag, regulations could hold the user accountable (Kiel and Solf 2017). To our knowledge, no judicial rules have been implemented in the regard of user-generated content in the United States, the FTC has, however, adapted the ‘Guides concerning Use of Endorsements and Testimonials in Advertising’ (FTC 2015). Overall, a further examination of the topic would be necessary not only for further research but also policymakers. Furthermore, positioning and formulation of the disclosures should be kept in mind, as studies point to the fact that the existing Facebook disclosures might not attract the attention they need (Wojdynski and Evans 2016). Interestingly, however, in line with Janssen et al. (2016), we found highly attentive participants to react less negatively to the post that included a disclosure. Consequently, we support Janssen and colleagues’ suggestion that a disclosure may backfire as it increases awareness for the brand, in our case the source of the post. Research on disclosures combined with a user source would again be of high interest. On the one side, the disclosure could inform the viewer of the persuasive intent. On the other side, viewers whose cognitive capacities are already depleted by the processing of the surrounding information might even remember more strongly that the source was a user. Another possible explanation lies in findings of the FTC regarding search engine advertisements and advertorials. The FTC found that participants spent significantly less time looking at native advertisements with a disclosure. However, this was not due to an increase in ad recognition (FTC 2017). Possibly, highly attentive INTERNATIONAL JOURNAL OF ADVERTISING 181 viewers’ identify advertisements regardless of the disclosure but in the case of a paid ad with disclosure, they just move on more quickly, and do not invest further resources on it. Limitations and future research This study investigated the interplay between two complex factors: the source of branded content on Facebook and viewers’ attention to it. Regarding the source manipulation, we have to mention several limitations. First, we exposed participants to posts of an unknown user. Yet, Facebook users are usually exposed to user-generated posts by their friends. As such, the effect of the user-generated alcohol posts could be even stronger. Future research should intend to include manipulation of weak versus strong ties with a user posting alcohol related brand content. Second, we excluded the branch of influencer marketing in this study, since it would have to be known by the participants that a specific source is not a regular user but an ‘influencer’. In this case, further aspects could confound the results, for instance effects triggered by the perception of the expert or celebrity status of the ‘influencer’ (e.g. Knoll and Matthes 2017). It could be of interest, however, to conduct follow-up studies on the characteristics of a user source, such as his or her popularity or expert status on a product. Third, we used alcohol as product category, as alcohol-related user-generated posts are very common in real-life (Nicholls 2012). It would definitely be worthwhile testing effects for other product categories. For instance, users often use Facebook as source for information on fashion trends (Newberry 2018). Concerning the manipulation of viewers’ attention, there are limitations as well. As much as we attempted to create a realistic browsing scenario, this experiment was still a laboratory study limited in external validity. Future studies should expose users to alcohol Facebook posts in a more naturalistic setting, such as on their own smartphone. Also, we worked with self-report data regarding the manipulation check of attention. Consequently, we cannot achieve the level of certainty regarding participants’ visual attention that an eye-tracking study would offer. Regarding cognitive involvement, we also relied on self-report date. In order to assess cognitive overload, however, further examinations are necessary that shed deeper insights into the role of attention on the processing of embedded advertising messages. Finally, we investigated this topic with a student sample. Even though student samples are often criticized for their lack of generalizability to the broad public (Kam, Wilking, and Zechmeister 2007), in our case a student sample is adequate and even necessary as they are the main target group of the examined advertising practice. Conclusion Our study suggests that persuasive messages on social media trigger users’ persuasion knowledge, if a brand is marked as their source. If, however, a user posts brand-related content, this can have persuasive effects without creating awareness for the persuasive potential. As it is only fair to the consumer to disclose covert advertising practices, we 182 M. MAYRHOFER ET AL. would argue for a stricter regulation of campaigns aiming at a broad user-participation. For instance, by also including ad disclosures on user-generated posts created in this context. As such, posts containing a hashtag or caption promoted by a brand could be marked as commercial content. Disclosure statement No potential conflict of interest was reported by the authors. Notes on contributors Mira Mayrhofer (PhD, University of Vienna) is a junior researcher at the University of Vienna, Vienna, Austria. Her research interests include covert persuasive techniques on television and social media as well as health communication. €rg Matthes (PhD, University of Zu €rich) is full professor of advertising research at the University Jo of Vienna, Vienna, Austria. His research interests include advertising effects, public opinion formation, and empirical methods. Sabine Einwiller (PhD, University of St. Gallen) is full professor of public relations research at the University of Vienna, Vienna, Austria. Her research interests include the effects of corporate communication, corporate social responsibility and crisis communication. Brigitte Naderer (PhD, University of Vienna) is a senior researcher at the University of Vienna, Vienna, Austria. Her research interests include persuasive communication, empirical methods, and advertising effects on children. References Alhabash, S., A.R. McAlister, E.T Quilliam, J.I. Richards, and C. Lou. 2015. Alcohol’s getting a bit more social: When alcohol marketing messages on Facebook increase young adults’ intentions to imbibe. Mass Communication and Society 18, no. 3: 350–75. Ashley, C., and T. Tuten. 2015. Creative strategies in social media marketing: An exploratory study of branded social content and consumer engagement. Psychology & Marketing 32, no. 1: 15–27. Boachie, P. 2018. User-generated content brings authenticity to brands. Adweek. https://www. adweek.com/digital/user-generated-content-brings-authenticity-to-brands/#. Boerman, S.C., and E.A. van Reijmersdal. 2016. Informing consumers about hidden advertising. A literature review of the effects of disclosing sponsored content. In Advertising in new formats and media: Current Research and implications for marketers, ed. P. De Pelsmacker, 115–46. London, England: Emerald. Boerman, S.C., E.A. van Reijmersdal, and P.C. Neijens. 2012. Sponsorship disclosure: Effects of duration on persuasion knowledge and brand responses. Journal of Communication 62, no. 6: 1047–64. Boerman, S.C., E.A. van Reijmersdal, and P.C. Neijens. 2015. How audience and disclosure characteristics influence memory of sponsorship disclosures. International Journal of Advertising 34, no. 4: 576–92. Brodmerkel, S., and N. Carah. 2013. Alcohol brands on Facebook: The challenges of regulating brands on social media. Journal of Public Affairs 13, no. 3: 272–81. Campbell, C., and N.J. Evans. 2018. The role of a companion banner and sponsorship transparency in recognizing and evaluating article-style native advertising. Journal of Interactive Marketing 43: 17–32. INTERNATIONAL JOURNAL OF ADVERTISING 183 Carah, N. 2014. Like, comment, share: Alcohol brand activity on Facebook. Deakin, ACT, Australia: Foundation for Alcohol Research & Education. https://espace.library.uq.edu.au/view/ UQ:329178. Carah, N., and M. Shaul. 2016. Brands and Instagram: Point, tap, swipe, glance. Mobile Media & Communication 4, no. 1: 69–84. Carpenter, S. 2018. Ten steps in scale development and reporting: A guide for researchers. Communication Methods and Measures, 12, no. 1, 25–44. doi:10.1080/19312458.2017.1396583 Choi, S. 2011. Guest editorial: Advertising and social media. International Journal of Advertising 30, no. 1: 11–2. De Jans, S., V. Cauberghe, and L. Hudders. 2018. How an advertising disclosure alerts young adolescents to sponsored vlogs: The moderating role of a peer-based advertising literacy intervention through an informational vlog. Journal of Advertising 47, no. 4: 309–25. Ertimur, B., and M.C. Gilly. 2012. So whaddya think? Consumers create ads and other consumers critique them. Journal of Interactive Marketing 26, no. 3: 115–30. European Commission. 2018. Behavioural study on advertising and marketing practices in online social media. https://ec.europa.eu/info/publications/behavioural-study-advertising-and-marketing-practices-social-media-0_en. Evans, N.J., and M.G. Hoy. 2016. Parents’ presumed persuasion knowledge of children’s advergames: The influence of advertising disclosure modality and cognitive load. Journal of Current Issues & Research in Advertising 37, no. 2: 146–64. Evans, N.J., J. Phua, J. Lim, and H. Jun. 2017. Disclosing Instagram influencer advertising: The effects of disclosure language on advertising recognition, attitudes, and behavioral intent. Journal of Interactive Advertising 17, no. 2: 138–49. Facebook. 2017. Facebok policies. https://www.facebook.com/policies/ads/#. Federal Trade Commission (FTC). 2013. Alcohol advertising. https://www.consumer.ftc.gov/ articles/0391-alcohol-advertising. Federal Trade Commission (FTC). 2015. Commission enforcement policy statement on deceptively formatted advertisements. https://www.ftc.gov/public-statements/2015/12/commissionenforcement-policy-statement-deceptively-formatted. Federal Trade Commission (FTC). 2017. Blurred Lines: An Exploration of Consumers’ Advertising Recognition in the Contexts of Search Engines and Native Advertising: A Federal Trade Commission Staff Report. https://www.ftc.gov/reports/blurred-lines-exploration-consumersadvertising-recognition-contexts-search-engines-native Fletcher, P. 2017. Report: Nearly 40% of publishers ignore FTC’s native advertising rules. https:// www.forbes.com/sites/paulfletcher/2017/03/19/nearly-40-percent-of-publishers-ignore-ftcs-nativeadvertising-rules/#3b43740067db. Fransen, M.L., E.G. Smit, and P.W. Verlegh. 2015. Strategies and motives for resistance to persuasion: An integrative framework. Frontiers in Psychology 6: 1201. Friestad, M., and P. Wright. 1994. The Persuasion Knowledge Model: How people cope with persuasion attempts. Journal of Consumer Research 21, no. 1: 1–31. Friestad, M., and P. Wright. 1995. Persuasion knowledge: Lay people’s and researchers’ beliefs about the psychology of advertising. Journal of Consumer Research 22, no. 1: 62–74. Fromm, J. 2016. The power of user-generated content and reviews stimulates millenial purchase behavior. Forbes. https://www.forbes.com/sites/jefffromm/2016/10/26/the-power-of-user-generated-content-and-reviews-stimulates-millennial-purchase-behavior/#2ddf4efa1394. Gomez-Rodriguez, M., K.P. Gummadi, and B. Schoelkopf. 2014. Quantifying information overload in social media and its impact on social contagions. In ICWSM (pp. 170–9). Ha, L. 1996. Advertising clutter in consumer magazines: Dimensions and effects. Journal of Advertising Research 36, no. 4: 76–85. Hayes, A.F. 2018. Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford Press. Hoffman, D.L., and Fodor, M. 2010. Can you measure the ROI of your social media marketing? MIT Sloan Management Review 52, no. 1: 41. 184 M. MAYRHOFER ET AL. Hwang, Y., and S.H. Jeong. 2016. ‘This is a sponsored blog post, but all opinions are my own’: The effects of sponsorship disclosure on responses to sponsored blog posts. Computers in Human Behavior 62: 528–35. Janssen, L., M.L. Fransen, R. Wulff, and E.A. Reijmersdal. 2016. Brand placement disclosure effects on persuasion: The moderating role of consumer self-control. Journal of Consumer Behaviour 15, no. 6: 503–15. Kam, C., J. Wilking, and E. Zechmeister. 2007. Beyond the ‘narrow data base’: Another convenience sample for experimental research. Political Behavior 29, no. 4: 415–40. Kiel, C. and P. Solf. 2017. Leifaden zur Kennzeichnung von Werbung auf Instagram. Die Wettbewerbszentrale. https://www.wettbewerbszentrale.de/media/getlivedoc.aspx?id¼35905. Kim, B.H., Y Pasadeos, and A. Barban. 2001. On the deceptive effectiveness of labeled and unlabeled advertorial formats. Mass Communication & Society 4, no. 3: 265–81. Kim, M., and D. Song. 2017. When brand-related UGC induces effectiveness on social media: The role of content sponsorship and content type. International Journal of Advertising 37, no. 1: 105–24. Knoll, J., and J. Matthes. 2017. The effectiveness of celebrity endorsements: A meta-analysis. Journal of the Academy of Marketing Science 45, no. 1: 55–75. Lang, A. 2000. The limited capacity model of mediated message processing. Journal of Communication 50, no. 1: 46–70. €derlund. 2015. Young consumers’ responses to suspected Liljander, V., J. Gummerus, and M. So covert and overt blog marketing. Internet Research 25, no. 4: 610–32. Lobstein, T., J. Landon, N. Thornton, and D. Jernigan. 2016. The commercial use of digital media to market alcohol products: A narrative review. Addiction 112, no. S1: 21–7. Matthes, J., and B. Naderer. 2016. Product placement disclosures: Exploring the moderating effect of placement frequency on brand responses via persuasion knowledge. International Journal of Advertising 35, no. 2: 185–99. Matthes, J., C. Schemer, and W.W. Wirth. 2007. More than meets the eye: Investigating the hidden impact of brand placements in television magazines. International Journal of Advertising 26, no. 4: 477–503. Mayrhofer, M., and B. Naderer. 2019. Mass media as alcohol educator for everyone? Effects of portrayed alcohol consequences and the influence of viewers’ characteristics. Media Psychology 22, no. 2: 217–43. Merckel, H. 2017. 3 tips for creating a social strategy fueled by user-generated content. Adweek. http://www.adweek.com/digital/harald-merckel-guest-post-user-generated-content/. Mittal, B. 1994. Public assessment of TV advertising: Faint praise and harsh criticism. Journal of Advertising Research 34, no. 1: 35–54. Moorman, C. 2016. Social media spending triples but falls short of expectations. Forbes. https:// www.forbes.com/sites/christinemoorman/2016/08/23/social-media-spending-triples-but-falls-shortof-expectations/#73f536a17257. Moriarty, E.S., and S. Everett. 1994. Commercial breaks: A viewing behavior study. Journalism Quarterly 71, no. 2: 346–55. Muntinga, G.D., M. Moorman, and E.G. Smit. 2011. Introducing COBRAs: Exploring motivations for brand-related social media use. International Journal of Advertising 30, no. 1: 13–46. Nairn, A., and C. Fine. 2008. Who’s messing with my mind? The implications of dual-process models for the ethics of advertising to children. International Journal of Advertising 27, no. 3: 447–470. Nelson, R.M., and J. Park. 2015. Publicity as covert marketing? The role of persuasion knowledge and ethical perceptions on beliefs and credibility in a video news release story. Journal of Business Ethics 130, no. 2: 327–41. Nicholls, J. 2012. Everyday, everywhere: Alcohol marketing and social media–current trends. Alcohol and Alcoholism 47, no. 4: 486–93. Newberry, C. 2018. A long list of Facebook statistics that matter to social marketers. Hootsuite. https://blog.hootsuite.com/facebook-statistics/. INTERNATIONAL JOURNAL OF ADVERTISING 185 Olenski, S. 2017. 4 ways brands should use native advertising in 2017. Forbes. https://www.forbes.com/sites/steveolenski/2017/02/16/4-ways-brands-should-use-native-advertising-in-2017/ #53e167b1c4c1. Park, W., and G. McClung. 1986. The effect of TV program involvement on involvement with commercial. In NA – Advances in consumer research, ed. R. Lutz, vol. 13, 544–48. Provo, UT: Association for Consumer Research. Pieters, R., and Wedel, M. 2007. Goal control of attention to advertising: The Yarbus implication. Journal of Consumer Research 34, no. 2: 224–33. Pinsky, I., A.R. Noto, M.C. Bote"quio de Moraes, E.L. Dos Santos, R. Sparks, and K. O’Brien. 2017. Alcohol industry sponsorship of university student sports clubs in Brazil. Journal of Studies on Alcohol and Drugs 78, no. 2: 306–12. Ridout, B., A. Campbell, and L. Ellis. 2012. ‘Off your Face (book)’: Alcohol in online social identity construction and its relation to problem drinking in university students. Drug and Alcohol Review 31, no. 1: 20–6. Smith, N.A., E. Fischer, and C. Yongjian. 2012. How does brand-related user-generated content differ across YouTube, Facebook, and Twitter? Journal of Interactive Marketing 26, no. 2: 102–113. Steyn, P., M.T. Ewing, G. Van Heerden, L.F. Pitt, and L. Windisch. 2011. From whence it came: Understanding source effects in consumer-generated advertising. International Journal of Advertising 30, no. 1: 133–60. Strick, M., R.B. van Baaren, R.W. Holland and A. van Knippenberg. 2009. Humor in advertisements enhances product liking by mere association. Journal of Experimental Psychology: Applied 15, no. 1: 35–45. Socialbakers. 2012. Alcohol brands shake up engagement. https://www.socialbakers.com/blog/ 1073-alcohol-brands-shake-up-engagement. Socialbakers. 2017. Facebook statistics – Brands. https://www.socialbakers.com/statistics/facebook/pages/total/brands/. Sung, Y., E. Kim., and S.M. Choi. 2018. # Me and brands: Understanding brand-selfie posters on social media. International Journal of Advertising 37, no. 1: 14–28. Tessitore, T., and M. Geuens. 2013. PP for ‘product placement’ or ‘puzzled public’? The effectiveness of symbols as warnings of product placement and the moderating role of brand recall. International Journal of Advertising 32, no. 3: 419–42. Thompson, V.D., and P. Malaviya. 2013. Consumer-generated ads: Does awareness of advertising co-creation help or hurt persuasion? Journal of Marketing 77, no. 3: 33–47. Truong, Y., and G. Simmons. 2010. Perceived intrusiveness in digital advertising: Strategic marketing implications. Journal of Strategic Marketing 18, no. 3: 239–56. van Reijmersdal, E.A., M.L. Fransen, G. van Noort, S.J. Opree, L. Vandeberg, S. Reusch, F. van Liesehout, and S.C. Boerman. 2016. Effects of disclosing sponsored content in blogs: How the use of resistance strategies mediates effects on persuasion. American Behavioral Scientist 60, no. 12: 1458–74. van Reijmersdal, E.A., N. Lammers, E. Rozendaal, and M. Buijzen. 2015. Disclosing the persuasive nature of advergames: Moderation effects of mood on brand responses via persuasion knowledge. International Journal of Advertising 34, no. 1: 70–84. Wei, M., E. Fischer, and K.J. Main. 2008. An examination of the effects of activating persuasion knowledge on consumer response to brands engaging in covert marketing. Journal of Public Policy & Marketing 2, no. 1: 34–44. Westgate, E.C., and J. Holliday. 2016. Identity, influence, and intervention: The roles of social media in alcohol use. Current Opinion in Psychology 9: 27–32. Wogalter, M.S., and K.R. Laughery. 1996. WARNING! Sign and label effectiveness. Current Directions in Psychological Science 5, no. 2: 33–7. Wojdynski, W.B., and N.J. Evans. 2016. Going native: Effects of disclosure position and language on the recognition and evaluation of online native advertising. Journal of Advertising 45, no. 2: 157–68. Zuwerink, J.J., and K.A. Cameron. 2003. Strategies for resisting persuasion. Basic and Applied Social Psychology 25, no. 2: 145–61. 186 M. MAYRHOFER ET AL. Appendix Attention Manipulation: Low: You will now see 20 Facebook Posts. Please browse through the posts, as you would do in your daily Facebook-Usage. Thank you! High: You will now see 20 Facebook Posts. Your task is to: Take as much time as you need to observe everything carefully Memorize all aspects attentively Look at all details in a very concentrated way This is very important, since you will be asked questions regarding your memory of the posts later on. Thank you for your effort! Manipulation check brand post (¼1) vs. paid advertisement post (¼2) Companies pay Facebook, so that I see post 2 (correct). I would only see post 1, if I had already liked the brand (correct). Every user could be exposed to post 2 in their newsfeed (correct). Post 1 and 2 were posted by a brand (correct). Post 2 is disseminated by a brand, without paying Facebook for it (false). There is no difference between post 1 and 2 (false). I would only see post 2, if I already know the brand (false). To reach new customers, a brand would use post 2 (correct). Received February 10, 2020, accepted March 16, 2020, date of publication March 19, 2020, date of current version March 30, 2020. Digital Object Identifier 10.1109/ACCESS.2020.2981892 Usability of Mobile Applications: A Systematic Literature Study PAWEä WEICHBROTH Department of Software Engineering, Faculty of Electronics, Telecommunications and Informatics, Gda´sk University of Technology, 80-233 Gda´sk, Poland e-mail: pawel.s.weichbroth@gmail.com ABSTRACT Since the release of the first mobile devices, the usability of on-board applications has been the concern not only of software vendors but hardware manufacturers as well. The academia community later willingly joined the discussion on usability in terms of theory and empirical measurement, having experience and knowledge in desktop settings. At first sight, such a background should guarantee a solid foundation to conduct research on software usability in a new setting. However, a preliminary study on the subject matter revealed methodological disorder in contemporary literature. As a matter of fact, a need emerged to review existing usability definitions, attributes and measures to recognize all associated aspects. In order to fill this void, we conducted a systematic literature review on usability studies indexed by the Scopus database and devoted to mobile applications. The input volume covers 790 documents from 2001 to 2018. The data analysis shows that the ISO 9241-11 usability definition has been adopted in an unchanged form and popularized as the standard by the HCI community. Secondly, in total, 75 attributes were identified and analysed. The most frequent are efficiency (70%), satisfaction (66%) and effectiveness (58%), which directly originate from the above definition. Subsequently, the less frequent are learnability (45%), memorability (23%), cognitive load (19%) and errors (17%). The last two concern simplicity (13%) and ease of use (9%). Thirdly, in the evaluation of usability, controlled observation and surveys are two major research methods applied, while eye-tracking, thinking aloud and interview are hardly used and serve as complementary to collect additional data. Moreover, usability evaluations are often confused with user experience dimensions, covering not only application quality characteristics, but also user beliefs, emotions and preferences. All these results indicate the need for further research on the usability of mobile applications, aiming to establish a consensus in the theory and practice among all interested parties. INDEX TERMS Mobile applications, usability, attributes, measures, usability evaluation methods, systematic literature review. I. INTRODUCTION Amobile application is defined as ‘‘a software application developed specifically for use on small, wireless computing devices, such as smartphones and tablets, rather than desktop or laptop computers’’ [1]. A recent Statista report shows that in 2017 smartphones had a share of 77% of the global mobile device market [2], and more than 32% of the global population used a smartphone [3]. Although technological progress has been made regarding mobile devices equipped with computing power, leading to a shift from desktop computers, many limitations and The associate editor coordinating the review of this manuscript and approving it for publication was Mario Luca Bernardi VOLUME 8, 2020 . challenges still remain [4]. From the many identified, usability has been the main concern, since the users of an application, and their judgment, ultimately decide on its success or failure [5]–[7]. Since the inception of the first smartphones, the subject of mobile application usability has gained attention both in academia communities and in the software vendors industry. While researchers are focused on formulating theories [8], modelling frameworks [9], and constructing methods and techniques [10], [11] for new settings, manufacturers simply desire to deliver high quality products [12]. Despite the abundance of research devoted to studies of mobile application usability on the one hand, and design patterns, prototyping tools and software frameworks on the This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 55563 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study other, the term tends to be vague and loose, weakening the ability to capture its real facets and impeding the construction of measures. As a consequence, such methodological disorder violates the core assumptions and principles laying beneath the foundations of the usability notion. Therefore, considering the need for the emergence of a usability definition, its attributes and measures, along with evaluation methods, valid for mobile applications, in this paper we made an attempt to find reliable answers by conducting a systematic literature review. We expect that the obtained results can be used not only by researchers to perform further studies in this area, but also for practitioners engaged in mobile application development and quality-in-use evaluation to better understand the characteristics and measures of the notion. The main contributions of this study include: (i) an evidence-based discussion of the usability definition, its attributes and measures, (ii) and an up-to-date map of the state of the art in usability evaluation methods (UEMs), adopted for and adapted to mobile applications, covering publications from 2001 to 2018. The rest of the paper unfolds as follows. Section 2 provides the background on the subject addressed, and related work. Section 3 describes the research methodology. The definition and execution of the literature review are respectively presented in Sections 4 and 5. Section 6 provides an analysis of the extracted data, while the results are further discussed in Section 7, along with the future research directions. The conclusions are raised in Section 8. II. BACKGROUND Most people tend to use products that are easy to understand, work as expected, and eventually deliver value. In the context of the software engineering, system usability plays the crucial role in shaping perceived quality in use by its users [13], [14]. Usability is the study of the intersection of between systems and users, tasks and expectations in the context of use. Since many software products have been determined to be insufficient to meet user needs, several comprehensive studies have been conducted so far under the term usability, which move towards a better understanding and relevant measurement, aiming to cover all valid phenomena in one framework or model [15]–[17]. The results of the study, introduced by Weichbroth [18], show that over time the definition of usability has evolved. In 1991 the Organization for Standardization (ISO), in response to the emergence of the need of the software community to standardize some facets of software products, publicized the 9126 standard, which defines usability as ‘‘a set of attributes of software which bear on the effort needed for use, and on an individual assessment of such use, by a stated or implied set of users’’ [19]. Then, in 1998, ISO refashioned the usability definition in the ISO 9241-11 norm, which actually states that usability is ‘‘the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’’ [20], [21]. 55564 While some argue that it is the most recognizable definition [18], others maintain that ‘‘a generally accepted usability definition still does not exist, as its complex nature is hard to describe in one definition’’ [22], [23]. The other usability definition can be found in ISO/IEC 25010 [24], which replaced the ISO/IEC 9126 standard from 2001 [25], and specifies usability as the ‘‘degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’’. Here, it is worth noting that these two latest paraphrased definitions, however differently particularized, still share exactly the same three virtues which affect the user’s ability to achieve specified goals. Since the inception of the first official usability definition, one might argue that a great plethora of usability attributes have been taken into consideration regarding the ability to use particular software products, ranging from monolithic systems to lightweight web pages. Having said that, based on the literature search and analysis, in view of usability attributes that contribute to the quality in use of the desktop software, the latest study [18] shows that the most frequent are efficiency, satisfaction, learnability and effectiveness. The least frequent are understandability and operability, memorability, errors, attractiveness and accessibility. To collect all necessary data in order to improve the quality of particular software facets, a variety of usability evaluation methods (UEMs) have been developed and empirically tested. One of the most recognized UEMs concern the family of user testing methods [26]–[28], in particular think-aloud protocol [29]–[31], question-asking protocol [32]–[34], performance measurement [35]–[37], log analysis [38]–[40], eye tracking [41]–[43], and remote testing [44]–[46]. Secondly, inspection methods, intended to be used by experts [47], refers to heuristic evaluation [48]–[50], cognitive walkthrough [51]–[53], perspective-based inspection [54]–[56], and guideline reviews [57]–[59]. Thirdly, inquiry methods, designed to gathering subjective data from users, utilize both quantitative (questionnaires [60]–[62]) and qualitative (interviews [63]–[65] and focus groups [66]–[68]) techniques. Furthermore, some authors also distinguish analytical modelling methods such as cognitive task analysis [69]–[71], task environment analysis [72]–[74] and GOMS analysis (Goals, Operators, Methods and Selection rules) [75]–[77]. Regarding the context of this study, Zhang and Adipat (2005) propose a generic framework for conducting usability tests for mobile applications through discussing existing methodologies and usability attributes [78]. As challenges, they point to the unique features of mobile devices and wireless networks which influence the usability of mobile applications, including mobile context, multimodality, connectivity, small screen size, different display resolutions, limited processing capability and power, and restrictive data entry methods. In the case of research methodologies for usability testing, they point to controlled laboratory experiments and field studies. While former limitations are ignorance of the mobile context and the preservation of reliable VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study network conditions and other environmental factors, then later, the lack of sufficient control over participants in a study, and dealing with issues such as the selection of environmental conditions, evaluation performance, data collection and condition control. They also identify nine generic usability attributes: learnability (ease of use), efficiency, memorability, errors, user satisfaction, effectiveness, simplicity, comprehensibility (readability) and learning performance. Hussain and Kutar (2009) introduce a hierarchical GQM (Goal Question Metric) model to evaluate mobile usability [79]. On the top level, they place three quality characteristics: effectiveness, efficiency and satisfaction. On the middle level, six guidelines are conceptualized: simplicity, accuracy, time taken, features, safety and attractiveness. Eventually, on the bottom, there is a mapping between questions and metrics, which enables the collection of quantitative data in order to evaluate usability. Kronbauer et al. (2012) propose a hybrid model for the evaluation of smartphone application usability [80]. In this study, the hybrid approach blends two methods for data capture, namely, Logging and ESM (Experience Sampling Method). The first one is based on data collection related to user interaction with an application. Using sensors available in smartphones for contextual data collection, such as luminosity intensity and the device’s position, allows the performance of statistical analysis regarding usability. The second one is based on the collection of users’ feelings towards a specific product through questions. These two methods are respectively used to measure efficiency, effectiveness and satisfaction. Harrison et al. (2013) developed the PACMAD (People At the Centre of Mobile Application Development) usability model, which identifies three major dimensions affecting the overall usability of a mobile application: the user, the task and the context of use [81]. However, the last one plays a crucial role, as an application may be used in multiple and very different contexts (e.g. environment, physical location, user’s state or activity performed). The model encompasses seven attributes, which together reflect the usability of an application: effectiveness, efficiency, satisfaction, learnability, memorability, errors and cognitive load. In some studies the model has been adopted to set up testing and evaluation frameworks [82], [83]. The novelty of the model concerns cognitive load as a new usability attribute. The authors claim that it can be observed that users of mobile applications often perform additional tasks, such as walking, while using the mobile device. For this reason, these additional tasks impact the user’s performance, arguing by example of a walking user who in the same time is texting a message which reduces walking speed as s/he is concentrating on typing (sending) the message. More recently, cognitive load has been acknowledged [84], or disregarded [85], as one of the usability notions. Actual usability, located in the frames of the quality-in-use model by Lew and Olsina (2013), comprises effectiveness, efficiency, learnability in use, and communicability [86]. VOLUME 8, 2020 They also emphasize the difference between the context of mobile applications and traditional, desktop or web applications. The context does not only concern hardware limitations (e.g. size of the screen), but also other factors, such as: user activity, day/time of day, location, user profile, device and network performance. Obviously, there are many more usability models, individually applicable to particular domains, such as mobile banking [87], or healthcare [88]; however, they were excluded from the discussion due to their specific attributes, classified as superior with respect to the others. III. RESEARCH METHODOLOGY A systematic literature review (SLR) in its nature differs from traditional narrative reviews by adopting a replicable, scientific and transparent process methodology. By design, it aims to reduce cognitive bias by providing an audit trail of the associated assumptions and procedures, reviewers decisions and conclusions on the one hand, and by identifying and documenting key scientific contributions to a field or question on the other. In order to provide a body of knowledge on the usability of mobile applications, we performed a systematic literature review by adopting and adapting the approach provided by Kitchenham and Charters [89], [90], since a large majority of the reported SLRs in software engineering has been carried out in respect to their guidelines [91]. According to the research design employed, this study consists of three steps, performed in a fixed sequence. Interdependency is revealed in the one-way output/input relations. Step 1 in the research methodology involves defining the research questions and the review protocol, which encompasses the data source and search strategy, the inclusion and exclusion criteria and the definition of the search string. The outcome of this step is described in Section 4. Step 2 in the research methodology involves executing the search string carried out on the database engine. Next, the obtained results are extracted and further processed. The outcome of this step is given in Section 5. Step 3 in the research methodology involves reviewing, analysing and reporting each data record, in order to consequently find and document answers for a defined set of the research questions. The outcome of this step is described in Section 6. IV. SYSTEMATIC LITERATURE REVIEW DEFINITION A. RESEARCH QUESTIONS DEFINITION Investigating the gap in usability between desktop and mobile settings, the following three questions arose: RQ1. How has usability for mobile applications been defined? RQ2. What are the usability attributes for mobile applications? RQ3. How have usability attributes for mobile applications been defined, and which measures and evaluation methods have been used? 55565 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 1. The general search query criteria. TABLE 3. The exclusion criteria (EXCLUDE) to the subject area (SUBJAREA). TABLE 2. The inclusion criteria (LIMIT-TO) to the subject area (SUBJAREA). These three interrogative statements provide the overall framework for conducting this study, by giving direction and setting up boundaries. B. DATA SOURCE AND SEARCH STRATEGY In line with the research methodology, step 1 involves a systematic search of the scientific literature on the topic of mobile application usability. Performed on Scopus, the largest abstract and citation database of peer-reviewed literature, counting over 71 million records [92], the search strategy aims at identifying indexed publications. A key issue when formulating a search strategy is to define the period of time to set up time boundaries. Being in our interest to obtain reliable and concise answers to the questions, we determined the closing date in December 2018. TABLE 4. The inclusion criteria (LIMIT-TO) for the document type (DOCTYPE). TABLE 5. The inclusion (LIMIT-TO) and exclusion (EXCLUDE) criteria for the language. C. SEARCH QUERY DEFINITION The search query was defined by the presence of ‘‘usability’’ and the string ‘‘mobile application’’ in titles, abstracts and keywords. These unique and specific terms, joined together in that order and in the extent of such meta-data, embody the authors’ common declaration of their research objectives and the adopted context of their performed studies. The summary, in terms of the search query construct, is given in Table 1. D. INCLUSION AND EXCLUSION CRITERIA In accordance with our research objective and questions, the first applied inclusion criterion relates to the subject area, which alternatively includes: computer science, engineering, mathematics, social sciences, or decision sciences. Table 2 presents the summary of the search query construct in this scope. In this study, usability is considered in the context of software, which is a concern of computer science and is also closely associated with the other abovementioned disciplines. In this line of thinking, we exclude irrelevant subject areas (e.g. Medicine, Health Professions, Chemistry and others). Table 3 depicts the summary of the search query construct in this scope. The second inclusion (exclusion) criterion was the document type which alternatively encompasses: conference proceedings, journal articles or book chapters. On the other 55566 hand, we did not take into account conference reviews and other reviews, which present non-scientific contributions. Table 4 outlines the summary of the search query in this scope. Not all scientists regard conference proceedings as a reliable and valuable source of knowledge. However, from our point of view, our judgement was not solely based on the document type, but on scrupulous reading and conscientious content analysis. The third inclusion (exclusion) criterion was the language, exclusively limited to English. Therefore, two other (Portuguese and French) were excluded. Table 5 depicts the summary of the search query construct in this regard. English has become the modern lingua franca in the modern world. The major international standardization bodies publish norms and standards in English, and communication channels between experts and communities follow the same rule as well. V. SEARCH EXECUTION A. SEARCH AND SELECTION In the first run, the search query (Table 1) produced 1,615 document results. To this volume, the inclusion VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study FIGURE 1. The distribution of the number of publications per year. and exclusion criteria were applied, defined respectively in Tables 2–5. The search strings, given in all these tables, were eventually combined by the relevant Boolean operators. The final search query construct, which entirely fulfils all the requirements, is given below. TITLE-ABS-KEY (usability AND ‘‘mobile application’’) AND (LIMIT-TO (SUBJAREA, ‘‘comp’’) OR LIMIT-TO (SUBJAREA, ‘‘engi’’) OR LIMIT-TO (SUBJAREA, ‘‘math’’) OR LIMIT-TO (SUBJAREA, ‘‘soci’’) OR LIMIT-TO (SUBJAREA, ‘‘deci’’) OR EXCLUDE (SUBJAREA, ‘‘medi’’) OR EXCLUDE (SUBJAREA, ‘‘heal’’) OR EXCLUDE (SUBJAREA, ‘‘ceng’’) OR EXCLUDE (SUBJAREA, ‘‘envi’’) OR EXCLUDE (SUBJAREA, ‘‘phys’’) OR EXCLUDE (SUBJAREA, ‘‘mate’’) OR EXCLUDE (SUBJAREA, ‘‘bioc’’) OR EXCLUDE (SUBJAREA, ‘‘ener’’) OR EXCLUDE (SUBJAREA, ‘‘psyc’’) OR EXCLUDE (SUBJAREA, ‘‘arts’’) OR EXCLUDE (SUBJAREA, ‘‘eart’’) OR EXCLUDE (SUBJAREA, ‘‘nurs’’) OR EXCLUDE (SUBJAREA, ‘‘chem’’) OR EXCLUDE (SUBJAREA, ‘‘neur’’) OR EXCLUDE (SUBJAREA, ‘‘econ’’) OR EXCLUDE (SUBJAREA, ‘‘agri’’) OR EXCLUDE (SUBJAREA, ‘‘immu’’) OR EXCLUDE (SUBJAREA, ‘‘phar’’)) AND (LIMIT-TO (DOCTYPE, ‘‘cp’’) OR LIMIT-TO (DOCTYPE, ‘‘ar’’) OR LIMIT-TO (DOCTYPE, ‘‘ch’’)) AND (EXCLUDE (PUBYEAR, 2019)) AND (LIMIT-TO (LANGUAGE, ‘‘English’’)) AND (EXCLUDE (LANGUAGE, ‘‘Portuguese’’) OR EXCLUDE (LANGUAGE, ‘‘French’’)) The results summary was checked in order to verify whether all the criteria were successfully applied. In total, the final search query eventually produced 887 documents, published between 2001 and 2018. The details of the volume data are as follows, while the numbers in brackets indicate the total number of publications: (a) published in English (887), VOLUME 8, 2020 (b) the subject area is from: computer science (803), decision sciences (40), engineering (198), mathematics (197) and social sciences (103), and (c) the document type is: conference proceedings (666), journal articles (196) or book chapters (25). The peak year is 2017 (140), followed by the years 2015 (110), 2018 (104) and 2016 (101), with an average of 74 documents published annually between 20082018 (Figure 1). The distribution of the number of publications increases in linear. However, in 2018 a fall was observed in comparison to the previous year, but still above the year 2016. The majority of documents were published by Springer in Lecture Notes in Computer Science, including sub-series Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics (148), while the largest contributor among journals is the Journal of Telecommunication, Electronic and Computer Engineering (12). The top three countries, the USA (136), Germany (81) and Malaysia (66), accounted for over 31% of the countries the authors were affiliated to. B. DATA EXTRACTION Having imported the reference data (authors, document title, year, and digital object identifier) to an external spreadsheet, we systematically searched for each record in full-text databases hosted by particular publishers and indicated as the source of the document. From the list of 887 records, in total 790 (89%) documents were fully available, while using a HAN system licensed account. To extract the data, three independent reviewing procedures were prepared and executed, respectively for each research question. In the first run, each available document was screened with the aim to identify and recognize a usability definition 55567 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 6. The list of adopted usability definitions for mobile settings. TABLE 7. The shares of adopted usability definitions for mobile settings. referenced by the author(s). The document was classified as relevant if: (a) usability, as a term, was explicitly defined and (b) correctly referenced. If the authors provided more than one definition and did not indicate a particular one as valid, then the first one given was assumed to be adopted. Eventually, 66 (8%) documents were classified as relevant as the input for analysis, with the aim of formulating an answer to the first research question. In the second run, each document was screened again to determine the overall quality and its relevance. A document was classified as relevant if: (a) the subject of the research was addressed to the usability of mobile applications, and (b) was not biased by a context of the research, such as: (i) application type or (ii) user-specific properties, such as: age, occupation, sex or (iii) system-specific support features, like visually impaired or disability. The review of the list produced 53 (7%) documents as relevant as the input for analysis with the aim of formulating an answer to the second research question. In the third run, the above list was reviewed and examined again with the aim of extracting attribute definitions, measures and UEMs. The document was classified as relevant if: (a) usability attributes being the subject of the study were explicitly defined, whereas a measure was valid if it captures the quantitative data which accurately describes one particular usability attribute. Ultimately, 39 (5%) documents were classified as relevant as the input for analysis with the aim of formulating an answer to the third research question. VI. DATA ANALYSIS This section addresses the analysis of the data extracted from the studies in accordance with the three defined 55568 research questions. We used a qualitative content analysis, which focuses on the characteristics of language as a communication channel, with attention to the specific subjects, narrowed and directed by particular research questions. RQ1. How has usability for mobile applications been defined? To this day, none of the authors have introduced any formal definition of usability associated with an application (system) running on a mobile device. Therefore, all identified and recognized definitions have been adopted from the existing general norms, standards and definitions. The great majority of authors (88%) have defined usability solely in terms of the ISO 9241-11 norm, while others have also made reference to ISO 25010 (4,5%) and ISO 9126 (3%) norms, as well as to the IEEE Glossary (1,5%), the Nielsen (1,5%) and Bevan (1,5%) definitions. Table 6 includes the full text of these six definitions, whereas Table 7 depicts findings of the shares of adopted usability definitions for mobile settings. RQ2. What are the usability attributes for mobile applications? In total, 75 usability attributes were identified and analysed. Among them, the most frequent are efficiency (70%), satisfaction (66%) and effectiveness (58%). Less frequent are learnability (45%), memorability (23%), cognitive load (19%) and errors (17%). The last two concern simplicity (13%) and ease of use (9%). The remaining attributes occurred four times or less. Table 8 outlines the details in this regard (the attributes which occurred only once are not included). VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 8. The list of adopted usability attributes for mobile settings. RQ3. How have usability attributes for mobile applications been defined, and which measures and evaluation methods have been used? The foremost attribute, efficiency is the ability of a user to complete a task with speed and accuracy. Efficiency is measured in a number of ways, such as the duration spent on each screen, the duration to complete a given task (a set of tasks), and the user’s error rate. Two evaluation methods are used: controlled observation and survey. Satisfaction is a user’s perceived level of comfort and pleasure, or a user’s perceived level of fulfilment of his expectations and needs. Satisfaction is measured only by using survey, with predefined statements with the Likert-scale rating system, which is typically used to capture a user’s intangible attitude towards an application. Effectiveness is the ability of a user to complete a task in a given context. It is measured by the number of successfully completed tasks, the number of steps required to complete a task, the number of double taps unrelated to the operation of an application, and the number of times that a back button is used by the mobile device (not the application). Learnability is defined twofold. First-time learnability refers to the degree of e...

Calculate your order
Pages (275 words)
Standard price: $0.00
Client Reviews
Our Guarantees
100% Confidentiality
Information about customers is confidential and never disclosed to third parties.
Original Writing
We complete all papers from scratch. You can get a plagiarism report.
Timely Delivery
No missed deadlines – 97% of assignments are completed in time.
Money Back
If you're confident that a writer didn't follow your order details, ask for a refund.

Calculate the price of your order

You will get a personal manager and a discount.
We'll send you the first draft for approval by at
Total price:
Power up Your Academic Success with the
Team of Professionals. We’ve Got Your Back.
Power up Your Study Success with Experts We’ve Got Your Back.
WeCreativez WhatsApp Support
Our customer support team is here to answer your questions. Ask us anything!
👋 Hi, how can I help?