Pagination is a common technique used to divide a large set of data into smaller, more manageable chunks. When interacting with the Modo API, we paginate our responses to make larger datasets more manageable and help improve the response time. In this article, we'll demonstrate how to handle these paginated responses in your code.
Below is an example of how you access the different pages within a response from the Modo API. The 'next' key in the JSON response enables you to access the next page of the response quickly.
#library imports
import pandas as pd
import requests
# url endpoint from Modo API
url = "https://api.modo.energy/public/v1/fpn'
# define parameters (payload)
params = {'date': '2022-02-01'}
# get token from https://platform.modo.energy/your-account/developers
headers = { 'X-Token': 'YOUR_API_TOKEN' }
# call Modo API
response = requests.get(url, headers=headers, params=params )
# First page of results
res = pd.DataFrame(response.json()['results'])
res
# pagination means we now limit each call to 1000 rows, but you can now get the next 1000 rows using the 'next' field of the API response:
response.json()['next']
'http://api.modo.energy/public/v1/uk_prices?date=2022-10-24&date_from=2021-10-01&date_to=2022-11-01&limit=1000&offset=1000'
# the example below shows a method to get all the data from all the pages # of a response quickly
df=pd.DataFrame()
while url is not None:
response = requests.get(
url,
params=params,
headers=headers
)
df = pd.concat([df,pd.DataFrame(response.json()['results'])])
url = response.json()['next']
If you've got any questions about connecting to the Modo API then please feel free to contact us - we're always happy to chat...