A Proposal for a Planetary-Scale Empirical Substrate
The Empirical Data Commons (EDC) is a proposal for a shared infrastructure to capture, structure, and preserve empirical observations as durable public artifacts.
It defines an empirical layer beneath existing scientific and technological systems—recording observational trace (sensor data, procedural context, and metadata) alongside interpretation, and making it available for replication, reanalysis, and reuse.
EDC is not a system for publishing conclusions. It is a framework for preserving what was actually observed.
Its aim is to address a structural gap: modern cognitive systems increasingly operate on compressed representations of reality, with limited access to the underlying empirical trace.
This article argues that modern cognitive systems operate on increasingly abstracted representations of reality, while lacking a shared, structured layer of empirical observation. It proposes that restoring this missing layer—by capturing, preserving, and making empirical trace reusable—is a necessary condition for reliable reasoning, reinterpretation, and discovery at scale.
Planetary-scale cognition is the distributed system of perception, inference, and action emerging across humans, machines, models, and institutions.
Its behavior is determined by its inputs. When those inputs are primarily symbolic—summaries, models, and inherited interpretations—rather than direct observation, the system becomes weakly grounded.
Machine-assisted cognition increases this distance. Decisions are made on compressed representations of reality, with limited access to the underlying empirical trace.
This introduces a structural risk: errors are not contained locally. They propagate across systems, are reinforced through reuse, and scale with the capabilities of the models acting on them.
The result is a cognitive layer that is highly capable, but increasingly detached from what was actually observed.
A shared, structured layer of empirical record does not currently exist. Without it, planetary-scale cognition operates on inherited conclusions rather than revisitable evidence.
The absence of a shared empirical substrate is not due to a lack of scientific activity, but a structural feature of how knowledge is produced.
What is typically preserved is not observation, but interpretation.
Scientific outputs prioritize compressed representations: statistical summaries, charts, models, and written conclusions. These are abstractions of an underlying event, shaped by experimental intent and human framing.
The raw observational trace — sensor data, procedural recordings, environmental context — is rarely retained or shared. In many cases, it is never captured in a durable, reusable form.
This creates a systematic substitution: symbolic outputs stand in for empirical reality.
The distinction matters. Interpretations are not reversible. Once observation is compressed into summary form, the original signal cannot be reconstructed or reinterpreted under new models or assumptions.
In earlier technological contexts, this trade-off was necessary. High-fidelity capture and storage of observation was expensive and operationally impractical.
That constraint no longer holds.
Modern sensors, storage, and bandwidth make it feasible to preserve observation at scale. What remains missing is not capability, but structure — systems that capture, organize, and expose empirical data as a first-class artifact.
Without such systems, the empirical layer continues to be discarded by default.
An experiment can be understood as a transformation across three layers of fidelity:
Layer 1: Sensorial Trace
Raw recordings of the physical world — video, sensor streams, timestamps, logs, and environmental context. This is direct observation.
Layer 2: Analytical Transform
Structured representations derived from the trace — graphs, tables, statistical summaries, and extracted features. This layer compresses observation to highlight patterns relevant to the experiment.
Layer 3: Linguistic Model
Narratives, conclusions, and formal claims expressed in natural language or symbolic form. This is the most human-legible layer, and the most widely distributed.
Each transition between layers involves lossy compression.
Multiple valid interpretations can be derived from the same observational trace. The form that is preserved reflects the assumptions, context, and priorities of the observer.
In practice, Layer 3 dominates. Most scientific outputs consist of linguistic models supported by selected analytical artifacts, with limited or no access to the underlying trace.
This creates a one-way transformation: from observation to interpretation, with no reliable path back.
Historically, this structure was driven by constraint. High-fidelity capture and storage of observation was expensive.
That constraint has shifted. Modern systems can capture and store continuous empirical trace at scale.
The bottleneck is no longer capture, but structure — classification, labeling, and contextualization of raw data.
This is now a technological problem, not a physical one.
The absence of a shared empirical substrate is not due to a lack of observation, but to how value is assigned and systems are structured.
Scientific and economic incentives favor outputs that are legible, compressible, and immediately useful: papers, models, metrics, and intellectual property. These are abstractions of observation, shaped for communication and short-term value.
Raw empirical trace does not fit these constraints. It is costly to capture, difficult to structure, and often slow to translate into recognized value. In some contexts, it exposes unclaimed insight, creating disincentives to share.
As a result, observation is selectively recorded, compressed, or not preserved at all.
This pattern extends beyond formal institutions. Across domains, individuals and small groups continuously generate empirical signal—through teaching, experimentation, construction, and fieldwork—but lack mechanisms to capture and contribute it in a structured, reusable form.
The limitation is infrastructural. There is no widely available system that allows contributors to:
The system reflects what it can ingest, not what exists. Large portions of empirical activity remain unrepresented—not because they lack value, but because they lack a path to inclusion.
Without a mechanism that assigns value to preservation and enables structured contribution, the empirical layer continues to be discarded by default.
When the observational trace is not preserved, the loss is not limited to replication. It eliminates the possibility of future reinterpretation.
Scientific outputs are typically symbolic compressions: charts, tables, and conclusions derived from an underlying event. These representations are shaped by the assumptions, methods, and priorities of the original observer.
Without access to the original trace, that framing becomes fixed.
Future analysts cannot:
This creates a form of epistemic loss that is largely invisible.
Entire classes of signal are discarded:
These losses are not exceptional. They are systemic.
Across domains, empirical work is routinely performed but not preserved in a reusable form:
What is lost is not just data, but optionality.
Without the base layer, past observation cannot be revisited as models improve. The system inherits conclusions, not evidence.
Preserving empirical trace is not about redundancy. It is about retaining signal that current models cannot yet interpret.
Without capture, there is no return path to the underlying event.
The Empirical Data Commons (EDC) is a proposed direction for establishing a shared empirical layer: a system in which observational trace is captured, structured, and preserved as a reusable resource.
It does not replace existing scientific or institutional systems. It addresses what they do not preserve: the underlying record of observation.
At minimum, such a system requires a small set of properties:
These properties define a substrate, not a specific implementation.
Implementations may vary, but systems that restrict access to empirical records or bind them to single interpretations limit their long-term reuse. In practice, this suggests architectures that support multiple independent hosts, replication, and shared access to empirical records.
If such a layer exists, additional capabilities become possible.
Empirical observation can become coordination-compatible: individuals or institutions can request specific measurements or experiments, and others can fulfill them by contributing structured observations. This enables new forms of capital allocation, where funding is directed toward the production of empirical data itself—for example, issuing open requests for measurements under defined conditions.
It also enables plurality at scale. The same experiment or observation can be performed repeatedly across different locations, times, and conditions, producing a distributed body of empirical records rather than a single canonical result. This supports comparison, variation, and reanalysis across contexts.
The role of an empirical commons is not to determine which interpretations are correct, but to ensure that the underlying observations remain available for verification, reinterpretation, and reuse as models and contexts evolve.
A further implication of preserving empirical trace is its effect on discovery.
Many findings originate not from new experiments, but from revisiting existing observations—particularly anomalies, failed trials, and configurations that were not considered significant at the time. In current systems, these are often unrecorded or discarded.
A structured empirical layer increases the density and availability of such observations. This enables reanalysis under new models, comparison across contexts, and the surfacing of patterns that were previously invisible.
Discovery, in this sense, is not only driven by novelty, but by the ability to return to what has already been observed.
This proposal identifies a missing layer, but does not resolve several core challenges:
These constraints do not negate the need for an empirical substrate, but they shape the conditions under which it can emerge.