dc.description.abstract | We have designed di erent heuristics for both searching on Massive graphs and
regularizing Deep Neural Networks in this work.
Both the problem of nding a minimum vertex cover (MinVC) and the maximum
edge weight clique (MEWC) in a graph are prominent NP-hard problems of great
importance in both theory and application. During recent decades, there has been
much interest in nding optimal or near-optimal solutions to these two problems.
Many existing heuristic algorithms for MinVC are based on local search strategies.
An algorithm called FastVC takes a rst step towards solving the MinVC
problem for large real-world graphs. However, FastVC may be trapped at local
minima during the local search stage due to the lack of suitable diversi cation
mechanisms. Besides, since the traditional best-picking heuristic was believed to
be of high complexity, FastVC replaces it with an approximate best-picking strategy.
However, best-picking has been proved to be robust for a wide range of
problems, so abandoning it may be a great sacri ce. Therefore, we rstly design
a diversi cation heuristic to help FastVC escape from local minima, and the proposed
solver is named WalkVC. Secondly, we develop a local search MinVC solver,
named NoiseVC, which utilizes best-picking (low complexity) with noise to remove
vertices during the local search stage in massive graphs. On the other hand, most
of existing heuristics for the MEWC problem focus on academic benchmarks with
relatively small size. However, very little attention was paid to solving the MEWC
problem in large sparse graphs. In this thesis, we exploit the so-called deterministic
tournament selection (DTS) heuristic for selecting edges to improve the local
search based MEWC algorithms.
Deep Neural Networks (DNN), have an extremely large number of parameters
comparing with traditional machine earning methods, su er from the the problem
of over tting. Dropout [Hinton et al., 2012, Srivastava et al., 2014] has been proposed
to address this problem. Dropout is an useful technique for regularizing and
preventing the co-adaptation of neurons in DNN. It randomly drops units with a
probability p during the training stage of DNN to avoid over tting. The working mechanism of dropout can be interpreted as approximately and exponentially combining
many di erent neural network architectures e ciently, leading to a powerful
ensemble. We propose a novel diversi cation strategy for dropout named Tabu
Dropout, which aims at generating more di erent neural network architectures in
fewer numbers of iterations. Besides, a recent work named Curriculum Dropout
achieves the state-of-the-art performance among the dropout variants by using a
scheduled p instead of a xed one. It gradually increases the dropping probability
from 0 to 1 p according to a time scheduling from curriculum learning. The
primary intuition is that dropout seems unnecessary at the beginning of training
and Curriculum Dropout starts training the whole neural networks without dropping,
which is called \starting easy". In this thesis, we design a new scheduled
dropout strategy using \starting small" instead of \starting easy", which gradually
decreases the dropping probability from 1 to p. We call this strategy Annealed
Curriculum Dropout.
Experiments conducted on related public standard datasets show that our proposed
heuristics for both searching on massive graphs and regularizing deep learning
have achieved better performance than the comparison methods. | |