English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
Add to Basket
 Local TagsRelease HistoryDetailsSummary
  Automatic Neural Network Architecture Optimization

Mounir Sourial, M. M. (2019). Automatic Neural Network Architecture Optimization. Master Thesis, Universität des Saarlandes, Saarbrücken.

Item is

 

show all hide all

Basic

show hide
Item Permalink: https://hdl.handle.net/21.11116/0000-0005-9C58-9 Version Permalink: https://hdl.handle.net/21.11116/0000-0007-64F3-6
Genre: Thesis

Files

show Files
hide Files
:
2019 MSC Thesis Maggie Sourial.pdf (Any fulltext), 3MB
 
File Permalink:
-
Name:
2019 MSC Thesis Maggie Sourial.pdf
Description:
-
OA-Status:
Visibility:
Restricted (Max Planck Institute for Informatics, MSIN; )
MIME-Type / Checksum:
application/pdf
Technical Metadata:
Copyright Date:
-
Copyright Info:
-
License:
-

Locators

show

Creators

show
hide
 Creators:
Mounir Sourial, Maggie Moheb1, Author
Weikum, Gerhard2, Advisor           
Cardinaux, Fabian3, Referee
Weikum, Gerhard2, Referee           
Yates, Andrew2, Referee           
Affiliations:
1International Max Planck Research School, MPI for Informatics, Max Planck Society, Campus E1 4, 66123 Saarbrücken, DE, ou_1116551              
2Databases and Information Systems, MPI for Informatics, Max Planck Society, ou_24018              
3External Organizations, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: Deep learning has recently become a very hot topic in Computer Science. It has invaded
many applications in Computer Science achieving exceptional performances compared
to other existing methods. However, neural networks have a strong memory limitation
which is considered to be one of its main challenges. This is why remarkable research
focus is recently directed towards model compression.

This thesis studies a divide-and-conquer approach that transforms an existing trained
neural network into another network with less number of parameters with the target of
decrasing its memory footprint. It takes into account the resulting loss in performance.
It is based on existing layer transformation techniques like Canonical Polyadic (CP) and
SVD affine transformations. Given an artificial neural network, trained on a certain
dataset, an agent optimizes the architecture of the neural network in a bottom-up man-
ner. It cuts the network in sub-networks of length 1. It optimizes each sub-network using
layer transformations. Then it chooses the most- promising sub-networks to construct
sub-networks of length 2. This process is repeated until it constructs an artificial neural
network that covers the functionalities of the original neural network.

This thesis offers an extensive analysis of the proposed approach. We tested this tech-
nique with different known neural network architectures with popular datasets. We
could outperform recent techniques in both the compression rate and network perfor-
mance on LeNet5 with MNIST. We could compress ResNet-20 to 25% of their original
size achieving performance comparable with networks in the literature with double this
size.

Details

show
hide
Language(s): eng - English
 Dates: 2019-05-152019-05-152019-05-152019-05-15
 Publication Status: Issued
 Pages: 76 p.
 Publishing info: Saarbrücken : Universität des Saarlandes
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: MounirMSc2019
 Degree: Master

Event

show

Legal Case

show

Project information

show

Source

show