Recursive Promises in Node

Recursive functions are extremely useful for many things in JavaScript. In React, you might want to traverse through all the children of a higher order component, even nested ones. There's many valid use-cases for these functions.

Last week I realized that I needed to recursively call a promise and eventually resolve at an unknown point. Why? In this case, I had to build a website scraper... I know, it's the best (not). I needed a way to keep clicking the next page link on a website until it no longer existed, and then resolve all URL's found on those pages.

Initially I was stuck because I couldn't find much help online. After a couple times of trial and error, I was overthinking it! Let's jump into the code:

const renderURLs = (site, url, existingData = []) => 
  new Promise(async resolve => {
    let response = await renderPage(site, url);
    let nextURL = getNextUrl(response.$);

    if (!nextURL) return resolve([...response.data, ...existingData]);
    else renderURLs(site, nextURL,…