Skip to main content

Programmatic Data Access in R

Reproducible programmatic data access in R, including caching downloaded files, dataRetrieval package, and building custom API requests with httr2.
apis
reproducibility
data-wrangling
lecture
intermediate
LearningLearning Objectives

After completing this session, you will be able to:

  • Explain why reading data from URLs is more reproducible than using hardcoded local file paths
  • Apply a file-existence cache pattern to avoid redundant downloads in R scripts
  • Use read_waterdata_monitoring_location() and read_waterdata_combined_meta() to discover USGS monitoring stations and available data
  • Retrieve daily water data for monitoring stations using read_waterdata_daily()
  • Visualize water temperature time series and seasonal patterns with ggplot2
  • Describe the client-server model and explain how HTTP APIs work conceptually
  • Construct a basic API request using httr2 (optional)

1 Overview

One of the most common reproducibility barriers in data-driven science isn’t in the analysis code itself, but rather in how data gets loaded in the first place. A script full of hardcoded file paths like read_csv("C:/Users/jsmith/Desktop/data_final_v3.csv") might work perfectly on one machine and fail immediately on every other.

Getting data programmatically – directly in your code, from a remote source – solves this problem. In this session, we’ll move from ad-hoc data downloads to systematic, reproducible data access. We’ll learn a caching pattern for downloading files reliably, then use the dataRetrieval package to retrieve and visualize USGS streamflow data from monitoring stations. Along the way, we’ll peek under the hood at how web APIs work.