Relaying Data through a Backend API
--
Avoiding Unnecessary Database Storage
When you create a backend API, it might feel like second-nature to persist any data you get from a 3rd party API that you want to use in your own app, and display on your frontend.
External API ---> Your API ---> Your Frontend
Take this really simple app I’m working on, where I fetch Mars Rover images to display through my React frontend:
For this, I fetch Mars Rover images from a NASA API through my own backend API (created with Ruby on Rails).
Why Relay Through Your Own API?
- Why use my own API? Well, it turns out that using an API Key securely with a React frontend might be a bit trickier than just using your own API to keep your keys secure.
- Also, I want to persist the data of NASA images if, and only if, you interact with the images, such as when you click the heart to show that you like a particular image, or you add a comment to the image.
- Further, and this is a BIG one:
There’s a scalability issue you run into if you save ALL of the images, or data that you gather from an external API.
- I don’t really have room to save every image I get from the NASA API, especially if I actually want to use this app frequently or for many users.
- So, I thought,
“I’ll make my API a ‘relay’ for the images until someone interacts with one, and in that case, I’ll persist it to my database.”
Planning Your Relay Flow
This requires planning for two things:
- Checking that the information isn’t already saved to your database. (Sometimes you do want to save the data, after all.) If it is saved, retrieve the persisted information.
- Formatting the data you are relaying so that it can easily be persisted in your database when you want to save it, and displayed on your frontend, just like any other information you’ve already saved.
Here’s the API class with the fetch_images method, which fetches Mars Rover Images from the NASA API, and the format_data method, which formats the relayed data to look like.
class Api < ApplicationRecord
@@base_url = 'https://api.nasa.gov/mars-photos/api/v1/rovers/'
@@rover_array = ["curiosity", "spirit", "opportunity"]
@@rover_max_sol = {"curiosity" => 3241, "spirit" => 2208,
"opportunity" => 5111}
@@camera_names = {
"FHAZ" => "Front Hazard Avoidance Camera",
"RHAZ" => "Rear Hazard Avoidance Camera",
"MAST" => "Mast Camera",
"CHEMCAM" => "Chemistry and Camera Complex",
"MAHLI" => "Mars Hand Lens Imager",
"MARDI" => "Mars Descent Imager",
"NAVCAM" => "Navigation Camera",
"PANCAM" => "Panoramic Camera",
"MINITES" => "Miniature Thermal Emission Spectrometer"
}def self.fetch_images(earth_date = nil)
# randomize rover
rover = @@rover_array[rand(2)]
max_sol = @@rover_max_sol[rover] if earth_date
date_query = "earth_date=#{earth_date}"
else
date_query = "sol=#{rand(max_sol)}"
end # Set data parameters to actual dates for opportunity rover in
earth_date
url = "#{@@base_url}#{rover}/photos?#{date_query}&api_key=#
{ENV["NASA_API_KEY"]}"
uri = URI(url)
resp = Net::HTTP.get(uri)
photos = JSON.parse(resp)
data = photos["photos"] # get more data if insufficient for frontend display
data = self.format_data(data, rover).compact if data.count < 15
data += self.fetch_images
end
data.shuffle
enddef self.format_data(data, rover)
data.map do |image_data|
if image_data["camera"] && image_data["earth_date"]
camera = @@camera_names[image_data["camera"]["name"]]
title = "#{rover.titleize} Rover—#{camera}—#{image_data["id"]}"
if saved_image = Image.find_by(title: title)
ImageSerializer.new(saved_image)
else
date = image_data["earth_date"]
{
image_url: image_data["img_src"],
title: title,
date_of_capture: date,
like_count: 0,
comment_count: 0
}
end
end
end
end
end
There are some extra things up there ^^ that are used to randomize which rover and which day is selected for images, as well as decoding the API data for the type of camera on a rover that took a particular image.
But, for the purposes of the relay example at hand, the important thing concerns the formatting of the data.
Data Formatting
if saved_image = Image.find_by(title: title)
ImageSerializer.new(saved_image)
else
date = image_data["earth_date"]
{
image_url: image_data["img_src"],
title: title,
date_of_capture: date,
like_count: 0,
comment_count: 0
}
endWhen Found in Our Database
If the image is already saved in the database, we can find it by its unique title, which is a combination of the rover name, the camera used to take the shot, and the image id stored in the NASA database:
title = “#{rover.titleize} Rover — #{camera} — #{image_data[“id”]}”
The ImageSerializer formats the data that is passed as a JSON object to our frontend like this:
class ImageSerializer < ActiveModel::Serializer
attributes :id, :image_url, :title, :date_of_capture, :like_count,
:comment_count
endWhen Relayed and Not Saved
If the image is not already saved in the database, we take each of the pieces of information we want to use, and which we might want to save in our database, and we return it in a hash structure that is identical to the hash-like JSON object structure we returned if the entry for this image already exists in our database:
date = image_data["earth_date"]
{
image_url: image_data["img_src"],
title: title,
date_of_capture: date,
like_count: 0,
comment_count: 0
}Note that the attributes here match those found for the saved image hash we returned above (and converted to a JSON object), with the exception of an “id,” since that only belongs to image data that has been persisted in our database, and is assigned at the creation of an image data’s entry into our database. So, we should not add an id when relaying an image that is not saved.
There certainly are other ways to do this, but this method seemed like a pretty straightforward way around saving every bit of information I gathered from the API and displayed to the user on through the frontend.
