Scholars’ Mine
Scholars’ Mine
Masters Theses
Student Theses and Dissertations
Fall 2019
Predictive modeling of webpage aesthetics
Predictive modeling of webpage aesthetics
Ang Chen
Follow this and additional works at: https://scholarsmine.mst.edu/masters_theses
Part of the Artificial Intelligence and Robotics Commons, and the Technology and Innovation
Commons
Department:
Department:
Recommended Citation
Recommended Citation
Chen, Ang, “Predictive modeling of webpage aesthetics” (2019). Masters Theses. 7913.
https://scholarsmine.mst.edu/masters_theses/7913
This thesis is brought to you by Scholars’ Mine, a service of the Missouri S&T Library and Learning Resources. This
work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the
permission of the copyright holder. For more information, please contact scholarsmine@mst.edu.
PREDICTIVE MODELING OF WEBPAGE AESTHETICS
by
ANG CHEN
A THESIS
Presented to the Faculty of the Graduate School of the
MISSOURI UNIVERSITY OF SCIENCE AND TECHNOLOGY
In Partial Fulfillment of the Requirements for the Degree
MASTER OF SCIENCE IN INFORMATION SCIENCE & TECHNOLOGY
2019
Approved by:
Dr. Fiona Fui-Hoon Nah, Advisor
Dr. Keng Siau
Dr. Langtao Chen
© 2019
Ang Chen
All Rights Reserved
iii
ABSTRACT
Aesthetics plays a key role in web design. However, most websites have been
developed based on designers’ inspirations or preferences. While perceptions of
aesthetics are intuitive abilities of humankind, the underlying principles for assessing
aesthetics are not well understood. In recent years, machine learning methods have shown
promising results in image aesthetic assessment. In this research, we used machine
learning methods to study and explore the underlying principles of webpage aesthetics.
Keywords: Aesthetics, Machine Learning, Webpage Aesthetics
iv
ACKNOWLEDGMENTS
This thesis is completed under the guidance of my committee members: Dr. Fiona
Fui-Hoon Nah, Dr. Keng Siau and Dr. Langtao Chen. They provided valuable guidance
and helped me get over difficulties. Their profound professional knowledge and rigorous
scientific attitude have a great impact on me. This thesis would not have been completed
without these amazing scholars.
I would like to express my most sincere and heartfelt gratitude to my advisor, Dr.
Fiona Fui-Hoon Nah. Dr. Nah contributed vast time and effort in helping me become a
researcher. She has been providing me with guidance and encouragement throughout this
thesis. Her knowledge, attitude and spirit has impacted me not only in research but also in
life. It is a wonderful experience doing research under her guidance.
At last, I would like to sincerely thank all professors for their hard work in
helping me finish this thesis. I am so fortunate to have the best family, professors and
friends who have been encouraging and supporting me through my master`s program.
v
TABLE OF CONTENTS
Page
ABSTRACT
……………………………………………………………………………………………………….. iii
ACKNOWLEDGMENTS ……………………………………………………………………………………. iv
LIST OF ILLUSTRATIONS
……………………………………………………………………………….. xii
LIST OF TABLES
……………………………………………………………………………………………… xii
SECTION
1. INTRODUCTION
………………………………………………………………………………………… 1
2. RELATED WORK……………………………………………………………………………………….. 5
2.1. WEBPAGE AESTHETICS …………………………………………………………………….. 5
2.2. AESTHETICS AND USER PREFERENCES …………………………………………… 6
2.3. COMPUTATIONAL INTERFACE AESTHETICS …………………………………… 7
2.4. TRADITIONAL WEBPAGE AESTHETIC ASSESSMENT
………………………. 8
2.5. WEBPAGE AESTHETIC ASSESSMENT USING DEEP
LEARNING………………………………………………………………………………………… 10
2.6. MODEL DICTIONARY ………………………………………………………………………. 11
2.6.1. Ordinary Least Squares Model.
…………………………………………………….. 11
2.6.2. Decision Tree Model. ………………………………………………………………….. 11
2.6.3. Random Forest Model. ………………………………………………………………… 12
2.6.4. Gradient Boosting……………………………………………………………………….. 14
2.6.5. Artificial Neural Network (ANN). ………………………………………………… 15
2.6.5.1. Architecture of artificial neural network (ANN). ………………….15
2.6.5.2. Input layer. ………………………………………………………………………16
vi
2.6.5.3. Hidden layer.
……………………………………………………………………17
2.6.5.4. Output layer. ……………………………………………………………………18
2.6.6. Deep Neural Network (DNN). ……………………………………………………… 19
2.6.7. Convolutional Neural Network (CNN). …………………………………………. 20
2.6.7.1. Convolutional layer…………………………………………………………..21
2.6.7.2. Pooling layer. …………………………………………………………………..22
2.6.7.3. Fully connected layer.
……………………………………………………….23
2.6.8. MobileNet.
…………………………………………………………………………………. 23
2.6.9. NasNet (Neural Architecture Search Network).
………………………………. 24
2.6.10. Inception Neural Network. …………………………………………………………. 24
2.7. AESTHETIC METRICS ………………………………………………………………………. 25
2.7.1. Color.
………………………………………………………………………………………… 26
2.7.1.1. W3C colors. …………………………………………………………………….26
2.7.1.2. Hue, saturation and value…………………………………………………..26
2.7.1.3. Colorfulness. ……………………………………………………………………26
2.7.2. Space-based Decomposition…………………………………………………………. 27
2.7.2.1. Number of leaves. …………………………………………………………….27
2.7.2.2. Number of image areas. …………………………………………………….27
2.7.2.3. Number of text groups. ……………………………………………………..27
2.7.2.4. Text area and non-text area.
……………………………………………….28
2.7.3. Quadtree Decomposition.
…………………………………………………………….. 28
2.7.3.1. Number of quadtree leaves. ……………………………………………….28
2.7.3.2. Symmetry.
……………………………………………………………………….28
vii
2.7.3.3. Balance……………………………………………………………………………28
2.7.3.4. Equilibrium. …………………………………………………………………….28
2.8. PERFORMANCE METRICS DICTIONARY
…………………………………………. 28
2.8.1. Mean Absolute Error. ………………………………………………………………….. 29
2.8.2. Mean Squared Error. …………………………………………………………………… 30
2.8.3. Root Mean Squared Error (RMSE). ………………………………………………. 30
2.8.4. R Squared (R2)……………………………………………………………………………. 31
3. METHODOLOGY
……………………………………………………………………………………… 33
3.1. ASSESSING AESTHETICS USING TRADITIONAL
METHODS
…………………………………………………………………………………………. 34
3.2. ASSESSING AESTHETICS USING DEEP LEARNING
MODELS……………………………………………………………………………………………. 35
3.3. RESEARCH METHODOLOGY …………………………………………………………… 36
3.4. DATASET ………………………………………………………………………………………….. 37
3.5. DATA COLLECTION PROCESS
…………………………………………………………. 38
3.6. DEALING WITH MISSING DATA
………………………………………………………. 39
3.7. DEALING WITH DUPLICATED DATA ………………………………………………. 40
3.8. DATA SPLIT
………………………………………………………………………………………. 40
3.9. FEATURE SCALING ………………………………………………………………………….. 41
3.10. STATISTICS OF PRE-PROCESSED DATA
………………………………………… 42
4. DATA ANALYSIS AND RESULTS ……………………………………………………………. 49
4.1. MODEL PERFORMANCE (AESTHETIC FEATURE METHOD). ………….. 49
4.1.1. Feature Selection. ……………………………………………………………………….. 50
4.1.2. Feature Selection Based on Interest. ……………………………………………… 51
viii
4.1.3. Feature Selection Based on Importance
(Using Random Forest Model). ……………………………………………………. 53
4.1.4. Model Performance on Selected Features
(Based on Importance).
……………………………………………………………….. 57
4.2. MODEL PERFORMANCE (DEEP LEARNING MODELS) ……………………. 59
4.2.1. Convolutional Neural Network with 2 Conv2D Layers.
…………………… 60
4.2.2. Convolutional Neural Network with 3 Conv2D Layers.
…………………… 63
4.2.3. Convolutional Neural Network with 4 Conv2D Layers.
…………………… 67
4.2.4. Convolutional Neural Network with 5 Conv2D Layers.
…………………… 71
4.2.5. NIMA NasNet Model.
…………………………………………………………………. 75
4.2.6. NIMA MobileNet Model……………………………………………………………… 81
4.2.7. NIMA Inception-ResNet-v2 Model. ……………………………………………… 84
4.3. REGRESSION ANALYSIS ………………………………………………………………….. 87
4.3.1. Analysis of Complexity.
………………………………………………………………. 87
4.3.1.1. Linear regression………………………………………………………………87
4.3.1.2. Locally weighted average scatterplot smoothing
(lowess).
…………………………………………………………………………..88
4.3.2. Analysis of Colorfulness. …………………………………………………………….. 89
4.3.2.1. Linear regression………………………………………………………………89
4.3.2.2. Locally weighted average scatterplot smoothing
(lowess).
…………………………………………………………………………..90
4.3.3. Why Some Models Have Better Performance. ……………………………….. 91
4.3.3.1. Non-linear relationship. …………………………………………………….91
4.3.3.2. Data noise. ………………………………………………………………………92
4.3.3.3. Over-fitting problem. ………………………………………………………..92
5. DISCUSSIONS ………………………………………………………………………………………….. 93
6. LIMITATIONS AND FUTURE RESEARCH ……………………………………………….. 95
ix
7. CONCLUSIONS ………………………………………………………………………………………… 97
APPENDIX
…………………………………………………………………………………………………………99
BIBLIOGRAPHY
………………………………………………………………………………………………108
VITA ………………………………………………………………………………………………………………..115
x
LIST OF ILLUSTRATIONS
Page
Figure 2.1. Basic Unit of an Artificial Neural Network — Artificial Neuron
(Vaibhav, 2018).
………………………………………………………………………………… 16
Figure 2.2. How a Biological Neuron Works (Wikipedia Contributors, 2019).
…………… 17
Figure 2.3. Artificial Neuron (Jayesh, 2018). …………………………………………………………. 18
Figure 2.4. A CNN Sequence to Classify Handwritten Digits (Sumit, 2018). …………….. 20
Figure 2.5. Inception Module, with Dimensionality Reduction
(Szegedy et al., 2015). ………………………………………………………………………… 25
Figure 3.1. Quadtree Decomposition (Reinecke et al., 2014).
…………………………………… 34
Figure 3.2. Space-Based Decomposition (Reinecke et al., 2014).
……………………………… 35
Figure 3.3. Statistics of W3C Color Features. ………………………………………………………… 43
Figure 3.4. Statistics of Other Aesthetic Features……………………………………………………. 44
Figure 3.5. Histograms of W3C Color Features. …………………………………………………….. 45
Figure 3.6. Histograms of Other Aesthetic Features………………………………………………… 46
Figure 3.7. Statistics of Average Aesthetic Rating. …………………………………………………. 47
Figure 3.8. Density Plot of Average Aesthetic Rating……………………………………………… 47
Figure 4.1. Machine Learning Model Performance using Aesthetic Features. ……………. 49
Figure 4.2. Important Features Selected by the Random Forest Model. …………………….. 54
Figure 4.3. Most Popular Colors around the World (William, 2015).
………………………… 55
Figure 4.4. Linear Regression Plots of ‘nonTextArea’, ‘textArea’, ‘blue’ and
‘complexitymodel’. ……………………………………………………………………………. 56
Figure 4.5. Regression Plots of ‘nonTextArea’, ‘textArea’, ‘blue’ and
‘complexitymodel’ with Fitted Regression Lines. ………………………………….. 57
Figure 4.6. Performance of the Models Using Features Selected by Random
Forest Model.
…………………………………………………………………………………….. 58
xi
Figure 4.7. Performance of Deep Learning Models. ……………………………………………… 59
Figure 4.8. Summary Information of the Convolutional Neural Network with
2 Conv2D Layers.
…………………………………………………………………………….. 60
Figure 4.9. Learning Curves of a CNN Model with 2 Conv2D Layers..
…………………… 62
Figure 4.10. Summary Information of the Convolutional Neural Network with
3 Conv2D Layers.
…………………………………………………………………………….. 64
Figure 4.11. Learning Curves of the CNN Model with 3 Conv2D Layers..
………………… 66
Figure 4.12. Summary Information of the Convolutional Neural Network with
4 Conv2D Layers.
……………………………………………………………………………. 68
Figure 4.13. Learning Curves of the CNN Model with 4 Conv2D Layers..
………………… 70
Figure 4.14. Summary Information of Convolutional Neural Network with
5 Conv2D Layers.
…………………………………………………………………………….. 72
Figure 4.15. Learning Curves of the CNN Model with 5 Conv2D Layers..
………………… 74
Figure 4.16. Examples of NIMA NasNet Predicting Aesthetics of Webpage.
…………….. 77
Figure 4.17. Learning Curves of NasNet Model Fine-tuned on Fully Connected
Layers.. …………………………………………………………………………………………… 80
Figure 4.18. Learning Curves of MobileNet Model Fine-tuned on Fully
Connected Layers.. …………………………………………………………………………… 83
Figure 4.19. Learning Curves of the Inception-ResNet-v2 Training on Fully
Connected Layers.. …………………………………………………………………………… 86
Figure 4.20. Scatter Plot with a Linear Regression Line of Complexity and
Aesthetic Rating. ……………………………………………………………………………… 88
Figure 4.21. Scatter Plot of Complexity and Aesthetic Rating using Lowess
Smooth Function ……………………………………………………………………………… 89
Figure 4.22. Scatter Plot with A Linear Regression Line of Colorfulness
(colofulnessmodelnewest) and Aesthetic Rating (mean_response). ………… 90
Figure 4.23. Scatter Plot of Colorfulness (colofulnessmodelnewest) and
Aesthetic Rating (mean_response) using Lowess Smooth Function ……….. 90
xii
LIST OF TABLES
Page
Table 4.1. Performance Scores of the Random Forest Model Using Only
Complexity and Colorfulness. …………………………………………………………….. 51
Table 4.2. Performance of Random Forest Using Features: Complexity,
Colorfulness, Quadratic Term of Complexity and Quadratic Term of
Colorfulness.
…………………………………………………………………………………….. 52
Table 4.3. Configuration of Compiler for the CNN Model with Two Conv2D
Layers. …………………………………………………………………………………………….. 61
Table 4.4. Evaluation Result of the CNN Model with Two Conv2D Layers.
……………. 63
Table 4.5. Configuration of Compiler for the CNN Model with Three Conv2D
Layers. …………………………………………………………………………………………….. 65
Table 4.6. Evaluation Results of the CNN Model with Three Conv2D Layers. ………… 67
Table 4.7. Configuration of Compiler for the CNN Model with Four Conv2D
Layers. …………………………………………………………………………………………….. 69
Table 4.8. Evaluation Result of Model CNN Model with Four Conv2D Layers. ………. 71
Table 4.9. Configuration of Compiler for the CNN Model with Five Conv2D
Layers. …………………………………………………………………………………………….. 73
Table 4.10. Evaluation Result of the CNN Model with Five Conv2D Layers. ……………. 75
Table 4.11. Evaluation Results of NIMA NasNet without Fine-tuning. …………………….. 76
Table 4.12. Compiler Configuration for the NasNet Model Fine-tuned on Fully
Connected Layers. …………………………………………………………………………….. 79
Table 4.13. Comparison of Evaluation Results of the NasNet Fine-tuned on Fully
Connected Layers and Part of the Convolutional Layers. ……………………….. 81
Table 4.14. Compiler Configuration of MobileNet Fine-tuned on Fully
Connected Layers. …………………………………………………………………………….. 82
Table 4.15. Comparison of Evaluation Results of the MobileNet Fine-tuned on
Fully Connected Layers and Part of the Convolutional Layers.
……………….. 84
Table 4.16. Compiler Configuration of the Inception-ResNet-v2. …………………………….. 85
xiii
Table 4.17. Comparison of Evaluation Results of the Inception-ResNet-v2
Fine-tuned on Fully Connected Layers and Part of the Convolutional
Layers. …………………………………………………………………………………………….. 87
1. INTRODUCTION
Aesthetics has been shown to have a dominant influence in e-commerce, such as
in influencing buyers’ purchase decisions (Postrel, 2001). Aesthetics also affects users’
evaluations (Tractinsky et al., 2000) and preferences (Schenkman and Jonsson, 2000).
Webpage design is an area that warrants attention to the assessments and dimensions of
aesthetics. A good webpage design can bring many benefits, such as increased click rates,
registration rates, subscription rates, volume of downloads, and conversions rates. From a
business point of view, these factors are critical to revenue. When a group of websites
offers similar services, users tend to choose sites that are visually more attractive than
others (Touch et al., 2012). Research shows that aesthetics in web design is a major
determinant of perceived credibility and trustworthiness (Fogg et al., 2003; McKnight et
al., 2002). Aesthetics has also been shown to positively influence behavior, such as user
performance and purchase intention (Reinecke et al., 2014; Moshagen et al., 2009; Bloch,
1995).
However, research on aesthetics is limited or lacking in the Human-Computer
Interaction (HCI) area. The traditional HCI field has been mostly concerned with
usability and functionality (Reinecke et al., 2014). Although designers are aware of the
importance of aesthetics, design decisions are made mainly based on “inspiration” and
“educated guesses” (Liu, 2003). It has also been shown that demographic differences,
such as personality, gender, and age, can have an impact on aesthetic impressions (Moss
& Gunn, 2009; Cyr et al., 2010; Reinecke & Bernstein, 2011; Wang, 2014).
2
Evaluating aesthetics is a highly subjective task. Different people can have
different views on aesthetics. A beautiful webpage in one’s eyes can be unpleasant in the
eyes of another. Sometimes, designers may develop websites which they believe to be
aesthetic but are not welcomed by users.
Most of the earlier research on aesthetics focus on design principles. The use of a
quantitative modeling approach to study aesthetics is fairly scarce in the literature.
Although the use of qualitative and interpretive approaches can provide deep insights into
aesthetics, it is often not easy to apply their findings in practice. From the perspective of
application, the use of quantitative methods could be easier to implement and test, and
hence, can generate greater practical value. Also, it has been argued that general design
principles may not apply in all contexts, especially from the perspective of aesthetics. On
the one hand, the design criteria proposed by qualitative researchers often cannot be
quantified or assessed quantitatively. On the other hand, the applicability of these criteria
is based on the context. Besides, people’s demographic differences and personal tastes
can also have an influence on aesthetic impressions (Lindgaard et al., 2006; Martindale et
al., 1990; Reinecke & Gajos, 2014).
With recent advances in machine learning and the boom and availability of data,
machine learning techniques have become a powerful and efficient tool in computer
vision. In the image recognition area, the convolutional neural network has been regarded
as the biggest advancement. It has helped researchers in understanding complex
phenomena and can potentially help us automatically assess aesthetics in HCI. In recent
years, deep learning techniques have been shown to be better in assessing aesthetic
quality of images than traditional methods that utilize webpage features associated with
3
aesthetics (Karayev et al., 2013; Lu et al., 2014, 2015a, 2015b; Dong & Tian, 2015; Kao
et al., 2016; Kong et al., 2016; Mai et al., 2016; Jin et al., 2016; Wang et al., 2016a,
2016b).
However, there are differences between image aesthetics and web aesthetics.
Image aesthetics are mostly used on photos and may not apply well to the area of web
aesthetics. Although a webpage can be turned into an image file with a screenshot, there
are many differences between the aesthetics of a webpage and an image. The composition
of a webpage is generally more complicated and sophisticated than a photo. It contains
many complex elements, such as text, borders, pictures, and even animations. Deep
learning models recognize and learn objects based on the edges of them. There are far
more edges between webpage elements in a screenshot than of a flower in a photograph.
Despite many differences between webpage aesthetics and image aesthetics, there could
be commonalities between them. According to a previous study (Lindgaard et al., 2006),
the aesthetic impression of a webpage can be formed within 50ms. A webpage can be
viewed as a whole image, and hence, we are interested to assess the performance of deep
learning models that have been used in image aesthetics and apply them in the context of
screenshots of webpages.
Traditional assessments of webpage aesthetics draw on specific measures of a
webpage to predict aesthetics. The advantage of this approach is that they can be
quantified and used directly in webpage design to guide web designers when a
comprehensive set of such measures is established in the literature. The disadvantage of
this approach is that identifying and determining these measures are often difficult,
require deep domain knowledge, and tend to miss other important measures. Further, the
4
extraction method of some measures is very complex, which is often difficult to apply
directly in rapid web evaluation applications.
In the deep learning area, Convolutional Neural Network (CNN) directly extracts
abstract measures from image input. The advantage of this method is that it can make out
key aspects of a picture. This approach effectively compensates for the shortcomings of
traditional aesthetic assessment methods by treating webpages as images in the web
assessment applications. While this approach sounds promising, it has its drawbacks. On
the one hand, the neural network is a black box, which means we have no way of
knowing how it rates a web page as beautiful or ugly. On the other hand, this method can
potentially only train a model to predict webpage aesthetics, and often fails to come up
with principles of aesthetics that the traditional approaches do. Whether one approach is
superior to another is still debatable. We can only say that each approach has its
advantages. However, attempts can be made to compare these approaches in assessing
and understanding webpage aesthetics.
In this thesis, we propose to use machine learning methods to study aesthetics.
First, we will review related work, which includes fundamentals about aesthetics and
machine learning methods used in this study. Second, we will describe the methodology
for the study. Third, we will present the data analysis and the results. Next, we will
present the shortcomings of the study and directions for future research. Finally, we will
summarize the results and provide conclusions for the thesis.
5
2. RELATED WORK
In this section, we will briefly introduce related work in the literature, which
includes webpage aesthetics, aesthetics and user preferences, computational interface
aesthetics, traditional webpage aesthetic assessment, and webpage aesthetic assessment
using deep learning methods.
2.1. WEBPAGE AESTHETICS
Aesthetics or beauty is one of the three basic requirements of architecture
according to Vitruvius, the first systematic theoretician of architecture (Kruft, 1994). The
three requirements are:
• Firmatis (Durability) – Architecture should be robust and remain in a good
condition.
• Utilitas (Utility) – Architecture should be useful and functioning well when
people are using it.
• Venustatis (Beauty) – Architecture should be delightful for people and their spirit.
Among the three requirements, the theory of Venustatis (or beauty) is more
complicated. Vitruvius believed that human’s “beauty” is the “truth of nature”, and
“nature’s designs” are harmonic and symmetric.
The terms, harmonic and symmetric, are closely associated with a term in
aesthetics–balance. Balance is an important principle in aesthetics. Balance means that
the visual weights of the page elements are evenly distributed throughout the page. When
6
a webpage is designed to be well balanced, users perceive equilibrium psychologically
(Lindgaard, 1999).
2.2. AESTHETICS AND USER PREFERENCES
Preferences are strongly influenced by aesthetics (Tractinsky, 2004). First
impressions are important. According to research, aesthetic impressions – the
spontaneous emotional responses based on visual preferences – can seriously influence
whether we perceive a product as useful or not (Sonderegger & Sauer, 2010). An
aesthetic impression is typically formed within 50 to 500ms during the first contact, and
it is persistent once it has been formed (Lindgaard et al., 2006; Tractinsky et al., 2006).
Because of these characteristics of aesthetic impressions, researchers often use static
screenshots of webpages to test whether users like those webpages (Reinecke et al.,
2013).
User preferences, as measured by evaluations of various aspects of a system or by
expressing attitudes towards the system, may not necessarily correspond to actual
decisions to use or buy one system over another (Ben-Bassat et al., 2006). Although a
user’s final choice is influenced by many other factors (e.g., economics, environment,
culture), the user’s preference based on its aesthetic impression is weighed heavily and is
hard to overcome in the decision process (Russo et al., 1998). If people develop an initial
preference for more attractive designs, judgments of objective measure information may
shift in the direction of more attractive products (Hoegg et al., 2010).
Although scholars have been trying to come up with universally applicable
principles of aesthetics, many factors can significantly impact these principles. For
7
example, due to individual and cultural differences, aesthetics could be perceived
differently by different people. Some people prefer concise designs while others prefer
designs involving artistic or special effects, which makes aesthetic assessments somewhat
subjective. Besides, websites frequently update their design styles to keep up with the
latest trends in website design. Websites today are very different from those in the 90s.
However, we do believe that it is useful to develop a better understanding of the
principles of webpage aesthetics. In addition, machine learning is powerful in learning
patterns based on data, which matches the characteristics and requirements of this
research.
2.3. COMPUTATIONAL INTERFACE AESTHETICS
Computational interface aesthetics is a field of study aimed at developing a
computation model for the aesthetic quality of interfaces. Past HCI studies have focused
on coming up with universal aesthetics principles, and considerations of computational
aesthetics have been largely ignored. Although aesthetic principles are useful in guiding
designers in their work, they do have limitations. For example, individuals can have
different aesthetic impressions when perceiving aesthetics (Lindgaard et al., 2006;
Martindale et al., 1990; Reinecke & Gajos, 2014). Thus, it is appropriate to personalize
design when targeting a specific group of users. More boldly speaking, if the technology
is mature enough, the design should modify and improve itself with each user.
It can be difficult to assess aesthetics objectively since aesthetics evaluation is
expected to be subjective. Different people may have different views on what aesthetics
is, which suggests that there will be large intra-class differences in aesthetic perceptions
8
(Jin, 2016). This problem is also challenging for machine learning algorithms since a
large intra-class variance will significantly bring down the performance of regression or
classification results.
The main challenge in the traditional aesthetics quantifying field is to evaluate
aesthetics from all aspects. Although more and more measures related to webpage
aesthetics are found, the significance of each measure is not the same. Researchers have
focused on two of the most striking measures: colorfulness and visual complexity
(Reinecke & Gajos, 2014).
Another challenge in computational interface aesthetics is that it requires
knowledge from multidisciplinary areas such as mathematics, computer science, human-
computer-interaction, art, design, psychology, and so on. Besides, researchers from
different knowledge domains prefer to explain aesthetics based on their respective
expertise, which makes it difficult to reach a consensus in aesthetic assessments.
2.4. TRADITIONAL WEBPAGE AESTHETIC ASSESSMENT
Using aesthetic measures has been the main approach in the area of computational
aesthetics. Researchers have been focusing on using aesthetic measures (i.e., aesthetic
rule-based features) to assess and explain aesthetics. According to Jin et al. (2016),
scholars in this area usually research using the following three steps:
1. Collect or design interfaces according to the needs of the research, then conduct
psychological experiments by having subjects assess the aesthetic quality of
interfaces. The results of the assessment are generally presented in the form of an
aesthetic score or class, such as “low” or “high”.
9
2. Craft measures or design principles such as rule of thirds, visual balance, and rule
of simplicity. Some researchers use generic image features such as low-level
image features, Fisher vectors, and bag of visual words to predict image
aesthetics. The source of these papers can be found in Jin et al.’s (2016) reference
section.
3. Build a statistical model or use machine learning models such as Support Vector
Machine and Random Forest to predict the aesthetic quality. The steps include
training the model based on the objective measures crafted and the subjective
aesthetic assessment collected, then use the model to predict the aesthetic quality
or help explore the relationship between the measures and aesthetic quality. These
methods are usually regarded as “white boxes” since they can explain or show the
relationship between the independent and dependent variables.
Measures of aesthetic dimensions or features are used as independent variables to
assess the dependent variable, aesthetic quality. Although aesthetics is a fairly subjective
concept, quantitative aesthetic measures do exist to quantify various aesthetic parameters
and measures of screen layouts.
Ngo’s theory (Ngo et al., 2003) is widely used as the baseline for measures of a
layout. There are 14 measures in Ngo’s work (2003): Balance (BM), Equilibrium (EM),
Symmetry (SYM), Sequence (SQM), Cohesion (CM), Unity (UM), Proportion (PM),
Simplicity (SMM), Density (DM), Regularity (RM), Economy (ECM), Homogeneity
(HM), Rhythm (RHM), as well as Order and Complexity (OM). The details of their
formulas and computations are described in their original work (Ngo et al., 2003).
10
Aesthetic measurement application (AMA), that was developed by Zain et al.
(2008) to automatically assess aesthetics, uses 6 measures, i.e., BM, EM, SYM, SQM,
RHM, and OM, that are based on Ngo’s work. The aesthetic scores which were given by
their model closely matched the rankings provided by the users. Altaboli and Lin (2011)
found that three measures, BM, UM, SQM, have significant effects on perceived
interface aesthetics. Maity and Bhattacharya (2017) found 9 measures, i.e., BM, CM,
EM, HM, PM, RM, SQM, SYM, and UM, to be statistically significant in predicting
aesthetics of webpages. However, they used a classification model instead of a linear
regression model. Thus, the weights of these measures are not known and numerical
evaluations cannot be offered by their study.
In our research, we will carry out a study of predictive modeling, linear and non-
linear regression analysis, and exploratory analysis. Since we will be using a number of
machine learning techniques and it is customary to refer to independent variables as
features in the machine learning terminology, we will refer to measures of aesthetic
dimensions as aesthetic features, or simply as features.
2.5. WEBPAGE AESTHETIC ASSESSMENT USING DEEP LEARNING
Boosted by the huge amount of data generated and vast improvement of
computation capability, deep learning techniques have been greatly improved. In recent
years, deep learning has gained tremendous success in computer vision, such as object
recognition, object detection, and image classification (Jin et al., 2016; Szegedy et al.,
2015; He et al., 2016).
11
However, deep learning techniques have rarely been used in the area of webpage
aesthetics. One related work is by Dou et al. (2019) who used a deep neural network
composed of 5 convolutional layers, 2 max-pooling layers, 2 fully connected layers, and a
regression layer to predict the aesthetic score of images of webpages. In addition, the
transfer learning technique was applied to increase the performance of the aesthetic
scoring model.
2.6. MODEL DICTIONARY
This thesis applies machine learning methods to assess the aesthetics of
webpages. Many scholars have applied these methods to natural photographs (Datta et
al., 2006; Wong & Low, 2009; Wu et al., 2010; Faria et al., 2013).
2.6.1. Ordinary Least Squares Model. The Ordinary Least Squares (OLS)
model is a statistical model of linear regression. It is used to estimate the relationship
between one or more independent variables and the dependent variable. The basic idea is
to minimize the sum of squares of the difference between the actual value and the model
predicted value of the dependent variable. The OLS model can search the best matched
parameters by minimizing the sum of squared errors. For a detailed introduction, please
refer to the work of Hutcheson (2011).
2.6.2. Decision Tree Model. The decision tree model is a tree-like model. The
‘tree’ consists of nodes and branches. There are two types of nodes: internal nodes and
leaf nodes. Each internal node represents a feature, and each leaf node represents a label
or target. Branches represent the output of that feature attribute on a range of values. For
details of decision trees, please refer to the paper by Quinlan (1986).
