After this lesson focused on accessing and using the ChatGPT Application Programming Interface (API), you should be able to:
- Explain what an API is and why it is useful
- Set up a user account and get an access token to the ChatGPT API
- Post a simple request and parse a response from the API using R
- Assemble, send, and parse more complex requests using R
- Access voice summary and translation functionality using R
1 Intro to ChatGPT API
1.1 Getting and storing an API key
- Generate a new API key using the instructions on the slides, through https://platform.openai.com/settings/organization/api-keys
- Save this key somewhere safe. Not in an R script or any Git-tracked file!
- Recommended: usethis::edit_r_environ() - opens .Renviron file. Edit this and save it!
- After you first set this up, you probably need to restart R to get it to kick in…
- Should look something like this:
OPENAI = your-API-key-here- To access in a script, use
Sys.getenv("OPENAI")(using the name you save it as)
api_key <- Sys.getenv("OPENAI")2 REST API - basics
REST APIs use “endpoints” - basically URLs that indicate the function or data you are trying to access. They are associated with a “method” (here we’ll focus on GET, to retrieve specific data, and POST, to send information and get a response)
2.1 Examples of APIs
2.2 How to access an API in R?
in R we’ll use the httr package, which has been around a while and is pretty straightforward, to create our API queries. Note, the developers have superseded httr with httr2 but since I’m not familiar with the new syntax, we’ll be old school for now! (watch here for updates)
3 Crafting a ChatGPT API call
First, a few terms:
- “tokens”: Tokens are the currency of these Large Language Models. A token is more or less one word, about 4-5 letters. Your prompt into ChatGPT is broken into input tokens, and the response you get is output tokens.
- “context window”: how much data the LLM can handle per request (both in and out) - kind of like the working memory. Older models could only “remember” a few sentences; newer models can handle a lot more.
- “token rate”: how rapidly can you feed data into/out of the LLM (again, both in and out) before you run into problems - again, older models were much more limited in their rates.
3.1 Endpoint URL
For basic text “completions” the endpoint URL is https://api.openai.com/v1/chat/completions.
There are other endpoints, for audio generation, audio transcriptions/translations, content moderation, image generation, etc.
3.2 POST body including model and message
Some things we need to specify for post body for the ChatGPT API:
- The
modelwe want to use! Tradeoff of cost vs. performance:- newer, more sophisticated models are much more expensive per token
- older models are more restrictive (how many tokens go in or out) but a lot cheaper
- let’s use
gpt-4o-mini: $0.15 per 1M tokens in, $0.60 per 1M tokens out
- The
message(s) to send, as a list. There are three “roles” for messages:role = "system"- how you want the model to answer questions, e.g., “You are a helpful and concise research assistant” or “You are a sassy teenager”role = "user"- this is where your query goesrole = "assistant"- you can use this to feed back the model’s prior responses, OR it can be helpful to “force” a certain kind of output… more on this later!
temperature(OPTIONAL) - this is a number from 0 to 1 indicating how much randomness you want the model to incorporate- closer to 0: more predictable, more focused, but less creative
- closer to 1: more creative, potentially less coherent or focused
max_tokens(OPTIONAL) - if you want to limit the response (e.g., to keep short responses and keep cost down), set a value here. Note, you can also tell thesystemrole to keep it brief!
3.3 Other metadata
We also need to supply the API with our key and our desired output format, using additional headers.
Authorization = 'Bearer <api key>',
Content-Type = 'application/json'4 Our first ChatGPT API call!
endpt <- 'https://api.openai.com/v1/chat/completions'
post_body <- list(
model = 'gpt-4o-mini',
messages = list(
list(role = 'system',
content = 'You are a helpful travel agent'),
list(role = 'user',
content = 'Tell me five things to do with my friends who are coming to
visit Santa Barbara - they love the outdoors and good food')),
temperature = 0.7
)
response <- POST(
url = endpt,
body = post_body,
encode = 'json',
add_headers(
Authorization = paste('Bearer', api_key),
`Content-Type` = 'application/json'
)
)That response is hard to understand. Use httr::content() to parse that response to make it sensible
x <- content(response, as = 'parsed')Now note that in this list of stuff, there is an element called $choices that contains $message, in which is $content that looks like an answer (and note $role is “assistant”). Let’s get that content!
out_text <- x$choices[[1]]$message$content
out_text5 Use cases for chat completions
5.1 Classification
Give ChatGPT a list of terms, and ask it to classify based on some criterion.
Let’s leave everything the same as the first query, just change the prompt to classify movies, feel free to try other classifications:
movies <- c('Die Hard', 'Gremlins', 'Star Wars', 'Halloween', 'The Princess Bride',
'Its a Wonderful Life', 'Shawshank Redemption', 'The Godfather',
'Jaws', 'Spirited Away', 'Alien', 'The Shining')
prompt <- paste('Classify these movies into Christmas vs not Christmas movies: ',
paste(movies, collapse = ', '))
post_body <- list(
model = 'gpt-4o-mini',
messages = list(
list(role = 'system', content = 'You are an opinionated movie critic'),
list(role = 'user', content = prompt)),
temperature = 0.7
)
response <- POST(
url = endpt, body = post_body, encode = 'json',
add_headers(
Authorization = paste('Bearer', api_key),
`Content-Type` = 'application/json'
)
)
out_text <- content(response, 'parsed')$choices[[1]]$message$content
out_text5.2 Sentiment analysis
Rate statements based on their sentiment, generally positive vs. negative or according to emotional weight.
reviews <- c('1. Best tacos ever',
'2. Service was very slow, but the food was great',
'3. Sick vibes and killer music',
'4. Ive had better')
prompt <- paste('Classify these restaurant reviews by sentiment: ',
paste(reviews, collapse = ', '))
post_body <- list(
model = 'gpt-4o-mini',
messages = list(
list(role = 'system', content = 'You are a sassy teen restaurant critic'),
list(role = 'user', content = prompt)),
temperature = 0.7
)
response <- POST(
url = endpt, body = post_body, encode = 'json',
add_headers(
Authorization = paste('Bearer', api_key),
`Content-Type` = 'application/json'
)
)
out_text <- content(response, 'parsed')$choices[[1]]$message$content
out_text6 Zero-shot vs one-shot vs few-shot prompting
In your prompt, you can provide one or more examples of expected output to point the LLM in a productive direction. Our previous classification and sentiment prompts did not provide any guidance on output, so these were zero-shot.
7 Single-turn tasks vs. conversations
As is, an API call is a standalone task, and once it is completed, it does not remain in memory. Until this point, we’ve been sending a single prompt to the API and accepting a single response. This is a “single-turn task” and if we were to send another prompt, the response to the previous prompt would not be remembered. But using the assistant role, we can set up a back and forth conversation between user and LLM.
7.1 Improved one- and few-shot prompting
One example for usage is to amplify a one-shot or few-shot prompt, by training the API as to the specific format of a response. In the responses to the restaurant reviews, the LLM really wants to give an explanation for its response, but perhaps we want to suppress the explanation just get the positive/negative/neutral answer.
sample_reviews <- c('1. Best tacos ever',
'2. Service was very slow, but the food was great')
sample_prompt <- paste('Classify these restaurant reviews by sentiment: ',
paste(sample_reviews, collapse = ', '))
sample_response <- '1. positive. 2. neutral.'
new_reviews <- c('1. Sick vibes and killer music',
'2. Ive had better')
new_prompt <- paste('Classify these restaurant reviews by sentiment: ',
paste(new_reviews, collapse = ', '))
post_body <- list(
model = 'gpt-4o-mini',
messages = list(
list(role = 'system', content = 'You are a sassy restaurant critic'),
list(role = 'user', content = sample_prompt),
list(role = 'assistant', content = sample_response),
list(role = 'user', content = new_prompt)),
temperature = 0.7
)
response <- POST(
url = endpt, body = post_body, encode = 'json',
add_headers(
Authorization = paste('Bearer', api_key),
`Content-Type` = 'application/json'
)
)
out_text <- content(response, 'parsed')$choices[[1]]$message$content
out_text7.2 Interactive mode
If you were creating a Shiny App, for example, where a user could write prompts to be sent to the API, and you wanted to allow for later prompts to incorporate answers from earlier prompts, you could set up a system to continually expand your POST body by appending the previous user prompts (as user) and API responses (as assistant). A while() loop would be a good structure for this, though for now, we will leave that as an exercise for the reader :)