English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Learning explanations that are hard to vary

Parascandolo, G., Neitz, A., Orvieto, A., Gresele, L., & Schölkopf, B. (2021). Learning explanations that are hard to vary. In Ninth International Conference on Learning Representations (ICLR 2021).

Item is

Basic

show hide
Genre: Conference Paper

Files

show Files

Locators

show
hide
Locator:
https://openreview.net/pdf?id=hb1sDDSLbV (Publisher version)
Description:
-

Creators

show
hide
 Creators:
Parascandolo, G, Author
Neitz, A, Author
Orvieto, A, Author
Gresele, L1, 2, Author              
Schölkopf, B3, Author              
Affiliations:
1Department High-Field Magnetic Resonance, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497796              
2Max Planck Institute for Biological Cybernetics, Max Planck Society, Spemannstrasse 38, 72076 Tübingen, DE, ou_1497794              
3Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1497647              

Content

show
hide
Free keywords: -
 Abstract: In this paper, we investigate the principle that good explanations are hard to vary in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.

Details

show
hide
Language(s):
 Dates: 2021-05
 Publication Status: Published online
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: -
 Degree: -

Event

show
hide
Title: Ninth International Conference on Learning Representations (ICLR 2021)
Place of Event: Wien, Austria
Start-/End Date: 2021-05-03 - 2021-05-07

Legal Case

show

Project information

show

Source 1

show
hide
Title: Ninth International Conference on Learning Representations (ICLR 2021)
Source Genre: Proceedings
 Creator(s):
Affiliations:
Publ. Info: -
Pages: - Volume / Issue: - Sequence Number: - Start / End Page: - Identifier: -