Database Design for Large-Scale, Complex Data

Written by:
Working Paper Number: SIPP-WP-100

High-dimensional data structures are the focus of this session.  We use the adjective complex in place of high-dimensionality because the problems that we describe arise both from the measurement of thousands of attributes and from the intricate logical conditioning of the measurement process.  Our paper provides answers to three questions associated with these data structures.

A significant structure always underlies data collected for scientific analysis.  The question is, How do we reveal that structure to support statistical analysis?  Time is an implicit dimension of a data structure.  The design of a data collection is not always identical over time.  Some of our discussion is devoted to how time is represented when measurements are asymmetric to different time points.

Page Last Revised - October 8, 2021