English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  DataJoint: Managing Big Scientific Data Using Matlab or Python

Reimer, J., Yatsenko, D., Ecker, A., Walker, E., Sinz, F., Berens, P., et al. (2016). DataJoint: Managing Big Scientific Data Using Matlab or Python. Poster presented at AREADNE 2016: Research in Encoding And Decoding of Neural Ensembles, Santorini, Greece.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/21.11116/0000-0000-7B76-2 Version Permalink: http://hdl.handle.net/21.11116/0000-0000-F374-B
Genre: Poster

Files

show Files

Locators

show
hide
Locator:
Link (Any fulltext)
Description:
-

Creators

show
hide
 Creators:
Reimer, J, Author
Yatsenko, D, Author
Ecker, A1, 2, 3, Author              
Walker, EY, Author
Sinz, F, Author              
Berens, P, Author              
Hoenselaar, A, Author
Cotton, RJ, Author
Siapas, AG, Author
Tolias, AS, Author              
Affiliations:
1Research Group Computational Vision and Neuroscience, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497805              
2Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497794              
3Department Physiology of Cognitive Processes, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497798              

Content

show
hide
Free keywords: -
 Abstract: The rise of big data in modern research poses serious challenges for data management: Large and intricate datasets from diverse instrumentation must be precisely aligned, annotated, and organized in a flexible way that allows swift exploration and analysis. Data management should guarantee consistency of intermediate results in subsequent multi-step processing pipelines such that changes in one part automatically propagate to all downstream results. Finally, data organization should be self-documenting to ensure that lab members and collaborators can access the data with minimal effort, even years after it was collected. While high levels of data integrity are expected, research teams have diverse backgrounds, are geographically dispersed, and rarely possess a primary interest in data science. While the challenges associated with large, complex data sets may be new to biologists, they have been addressed quite successfully in other contexts by relational databases, which provide a principled approach for effective data management. DataJoint is an open-source framework that provides a clean implementation of core concepts of the relational data model to facilitate multi-user access, effcient queries, distributed computing, and cascading dependencies across multiple data modalities. Critically, while DataJoint relies on an established relational database management system (MySQL) as its backend, data access and manipulation are performed transparently in the commonly-used languages MATLAB or Python, and DataJoint can be integrated into new and existing analyses written in these languages with minimal effort or additional training. DataJoint is not limited to particular file formats, acquisition systems, or data modalities and can be quickly adapted to new experimental designs. DataJoint and related resources are available at http://datajoint.github.com.

Details

show
hide
Language(s):
 Dates: 2016-06
 Publication Status: Published in print
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: ReimerYEWSBHCST2016
 Degree: -

Event

show
hide
Title: AREADNE 2016: Research in Encoding And Decoding of Neural Ensembles
Place of Event: Santorini, Greece
Start-/End Date: -

Legal Case

show

Project information

show

Source 1

show
hide
Title: AREADNE 2016: Research in Encoding And Decoding of Neural Ensembles
Source Genre: Proceedings
 Creator(s):
Affiliations:
Publ. Info: -
Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 99 - 99 Identifier: -