Surface Observation Pipeline Research
Proposal · Pre-data collection

Study 01: How Quickly Does Weather Data Reach the Public?

Proposed independent study Educational outcome: an accessible explanation of how observations travel

The Question Behind This Study

When someone glances at a weather app at 7:15 PM, the temperature shown isn't always from 7:15 PM — it might be from 6:54 PM or earlier, depending on which channel the app reads from and how recently that channel updated. The "freshness" of weather information varies in ways most people never see. This study aims to document, in clear and reproducible terms, how long an observation actually takes to travel from the sensor at a major US airport to the consumer-facing weather products people see.

Background

Each major US city has an automated surface weather sensor that records temperature, wind, pressure, and visibility on a regular schedule. From the moment of measurement, the observation passes through NOAA's processing pipeline and emerges on several public information channels — each maintained for a different audience and updating on its own rhythm. NOAA's Synoptic Data documentation describes high-frequency observation availability as typically arriving "between 2 and 5 minutes" after the observation [1]. The Aviation Weather Center service refreshes its data approximately once a minute [2]. NOAA's Telecommunications Gateway file distribution updates approximately every five minutes [3].

Those rhythms are documented in technical specifications, but their practical day-to-day behavior — how consistent the timing actually is, how often a particular channel falls behind, what time-of-day patterns exist — is rarely characterized for individual cities in publicly accessible writing. The aim of this study is to produce that characterization in a form a general reader can understand and a science journalist can cite.

Surface observation dissemination pipeline A diagram showing a single observation generated at 54 minutes past the hour, then propagating through four parallel channels (LDM push, public JSON API, file-based gateway, community archive) before arriving at a consumer. Automated Sensor Observation @ XX:54 MADIS LDM (push) Real-time delivery aviationweather.gov JSON API · 60-sec cache Telecommunications Gateway tgftp.nws.noaa.gov · 5-min Iowa Environmental Mesonet Community HTTP archive Δt₁ ? Δt₂ ? Δt₃ ? Δt₄ ? Consumer Records arrival time
Figure 1. Conceptual data flow for a single weather observation through four public NOAA dissemination channels. The latency Δt for each channel — from observation time to consumer-side availability — is the quantity this study proposes to measure empirically.

Research Questions

  1. From the moment a sensor at a major US city records an observation, how long does it take for that observation to be available through each of the public NOAA channels? What is the typical case, and what are the worst cases?
  2. Do these timing patterns differ from city to city? From morning to afternoon? Between weekdays and weekends?
  3. How often does an individual city experience an unusually long delay — a stall — and what does this look like when it happens?
  4. When two channels both eventually deliver the same observation, what's the typical timing relationship between them?

Proposed Approach

Observations from approximately 10–15 major US cities will be collected concurrently from each public channel, with care taken to record the moment each observation arrives at the collection point. The collection runs continuously for a period sufficient to characterize day-to-day variation, after which the resulting dataset is analyzed and the findings are written up in plain language with supporting visualizations.

Who Benefits From This Work

A clear, public-facing characterization of how weather observations move through NOAA's information channels is useful in several practical ways:

Anticipated Findings

Cross-References to Other Studies

References

  1. Synoptic Data PBC. High Frequency ASOS — Documentation. docs.synopticdata.com
  2. NOAA Aviation Weather Center. Data API. aviationweather.gov/data/api
  3. NOAA NWS. Telecommunications Gateway Data Help. weather.gov/tg/datahelp
  4. NOAA NWS. MADIS — METAR Data. madis.ncep.noaa.gov
  5. Iowa Environmental Mesonet. METAR datasets. mesonet.agron.iastate.edu