English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Learning explanations that are hard to vary

Parascandolo, G., Neitz, A., Orvieto, A., Gresele, L., & Schölkopf, B. (submitted). Learning explanations that are hard to vary.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/21.11116/0000-0006-F4F2-5 Version Permalink: http://hdl.handle.net/21.11116/0000-0006-F4F3-4
Genre: Paper

Files

show Files

Locators

show
hide
Locator:
https://arxiv.org/pdf/2009.00329.pdf (Any fulltext)
Description:
-

Creators

show
hide
 Creators:
Parascandolo, G, Author
Neitz, A, Author
Orvieto, A, Author
Gresele, L1, 2, Author              
Schölkopf, B3, Author              
Affiliations:
1Department High-Field Magnetic Resonance, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497796              
2Max Planck Institute for Biological Cybernetics, Max Planck Society, Spemannstrasse 38, 72076 Tübingen, DE, ou_1497794              
3Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1497647              

Content

show
hide
Free keywords: -
 Abstract: In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.

Details

show
hide
Language(s):
 Dates: 2020-09
 Publication Status: Submitted
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: -
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show