Search API Connector Documentation

Print

Scheduling

premium

Automated scheduling for the API Connector add-on for Google Sheets allows you to refresh your reports automatically at defined intervals.

Contents

Create a Trigger

To create an automatic trigger, click the Schedule tab in the top navigation menu in the API Connector sidebar and click Create trigger. Optionally, tick the box to create a log of all requests run in your sheet.

triggers-add new

Trigger Settings

When you add a new trigger there are several available options.

triggers-settings
  • Active/Paused. Paused triggers will not run until their status is switched to active.
  • API Request name. Here you can choose specific API request names from the drop-down menu, so different requests can have different triggers. Alternately, you can choose the ‘All’ option to select all of them at once, but be mindful of Google’s limits.
  • Run request. Once you’ve chosen which request(s) you’d like to schedule, choose your desired refresh frequency.
  • Run as. By default, this will be set to “Any user”, in which case triggers may run as any user who:

    • a) has Edit access to the sheet,
    • b) has an API Connector account that enables scheduling, and
    • c) has opened the sidebar in the sheet at least once

      In some cases, you may wish to assign scheduled requests to a specific user, for example if only that user has authenticated an OAuth connection, or you want to split up requests between users to avoid hitting Google’s limits.

  • Trigger name. Give your trigger a name.

Available Refresh Frequency

Currently reports can be updated at the following intervals:

  • hourly, including every 1, 3, 6, and 12 hours (Business / Team plans only)
  • daily
  • weekly
  • monthly

The exact execution time is set by Google, see here for more info.

Trigger Execution Time

Triggers are set within API Connector and then managed by Google, which will execute your requests on a recurring interval.

According to Google, the time is slightly randomized — for example, if you set a daily trigger between 2am – 3am, Google will choose a time between 2am and 3am for your trigger. Following that, your trigger time will remain consistent each day, such that exactly 24 hours passes before the trigger fires again.

If you want to know the exact time your triggers execute, you can set up logging or add a timestamp to your request (located under Output Options).

Trigger Ordering

In some cases you may want one request to run before a second request, to ensure that some ID or other value is ready for a subsequent request. This can be done by ordering triggers. There are two methods of ordering triggers:

  1. If you select an “All” trigger, it will run through every request in the sheet, in the order they’re set in the Requests screen. Requests will run from top to bottom.
  2. If you select specific requests instead of All, it will run through them in the order they’re set on the Trigger screen. Again, triggers will run from top to bottom.

This means that if two triggers are set to run at the same schedule, the one higher up will run prior to the other. Further, if two triggers are set at different intervals, the one higher up still takes precedence over the other one.

Example

In the screenshot below, Daily will always run first. Even if Hourly runs every hour while Daily runs only once a day, Daily will run first during the hour that they both run.

scheduling-img5

Re-order Triggers

On both the Requests screen and the Saved Triggers screen, triggers can be re-ordered by grabbing their handles and dragging and dropping them to the desired order.

Trigger Timezones

All triggers will use the spreadsheet timezone, which you can view by clicking File > Spreadsheet Settings. For more information on changing your sheet timezone, check https://support.google.com/docs/answer/58515.

Please note that if you change your spreadsheet timezone, you will need to delete and re-add your triggers to run off the updated timezone.

Trigger Limits

  • Google doesn’t allow Workspace add-ons to run triggers more than once an hour. This is their policy for all add-ons using their time-driven triggers, as detailed in their documentation.
    api-connector-triggers2
    As an alternative, you may be able to use API Connector’s custom IMPORTAPI function to force more frequent refreshes (check the section on Fast cell-based refresh). Note that IMPORTAPI is not the same as scheduling and is generally only used when you are active in your sheet.
  • Google Apps Script services imposes additional quotas and limits (info). For example there is a limit of 20,000 URL fetch calls per day per user, processes can only run for 6 minutes each hour, and triggers have a total runtime limit of 90 mins a day. If your triggers stop running or get delayed, you are likely running into these limits.
    api-connector-triggers-img3
  • Google allows their scheduler to run in up to 300 sheets per script. This means that if you have 300 sheets, your API Connector triggers won’t run in the 301st sheet.

Troubleshooting Triggers

The most common causes of trigger issues are the 6 minutes/hour and 90 minutes/day runtime limits on triggers shown above. These runtimes are shared between all your sheets that use triggers, so if you have many sheets or queries refreshing at the same time, your triggers may instead be skipped or pushed to the next hour or day. This can happen consecutively, such that your triggers don’t run at all.

If triggers time out, you will see a “Request started” message without a subsequent “Request completed” message in the logs.

If you are experiencing issues with your triggers, please try the following tips to resolve them.

Reduce data and spread out triggers

  1. Reduce the size of your request. Most APIs provide some way to limit the response through parameters in the URL. Filtering fields through the visual field editor or JMESPath can help too as it reduces the time required to print data into the sheet.
  2. To avoid inadvertently reaching Google’s limits, limit triggers and queries where possible. Reserve hourly refreshes for your most important queries, and don’t run hourly refreshes if you only need daily. Your data may never fully refresh if you implement large numbers of hourly triggers.
  3. Spread out your requests, i.e. instead of scheduling them all to run at midnight, set some at midnight, some at 1am, some at 2am, and so on.
  4. Be aware that paginated calls each count as one URL fetch request, i.e. if you run a request that paginates through 100 pages, Google will count 100 urlfetches. Large amounts of pagination are also more likely to run into Google’s 6 minutes per execution limits. Consider using append mode rather than re-fetching an entire data set via pagination.

Address authorization-related errors

By default, triggers will automatically run under any user who meets these conditions:

  • a) has Edit access to the sheet,
  • b) has an API Connector account that enables scheduling, and
  • c) has opened the sidebar in the sheet at least once

This can lead to two authorization-related issues:

  1. For OAuth requests, if multiple users meet the above conditions, the “wrong” (i.e. unauthenticated) user may execute the trigger. Therefore, make sure you’ve either selected an authenticated user under the Run as setting, or that all users have authenticated themselves to any OAuth connections.
  2. When using the Run as setting, make sure that user meets the conditions listed above. For example, triggers will fail if they are assigned to a user who has never opened the API Connector sidebar in the sheet.

Other tips

  1. Remove unecessary triggers. Navigate to https://drive.google.com/drive/my-drive and click Recent to identify files that have updated recently, and check to see if these updates were automatically triggered by any script or add-on (not just API Connector). Remove any unused triggers, as they may be interfering with the triggers you want.
  2. Limit usage of functions like IMPORTAPI(), IMPORTJSON(), and GOOGLEFINANCE() as they can run urlfetches every minute or two and expend your quota.
  3. Triggers in the Trash bin will remain active until they are permanently deleted. Therefore, navigate to https://drive.google.com/drive/my-drive and click Trash > Empty Trash to remove them permanently.
  4. Occasionally, Google deletes or disables triggers without apparent reason or warning. This is a bug that has been reported and acknowledged by Google, e.g. here and here. (Triggers are stored on Google’s side, so you can determine if you’ve been affected by setting triggers and then looking to check if they’ve been deleted or disabled. You can see all triggers associated with your account here: https://script.google.com/home/triggers).
  5. Re-opening the add-on automatically re-initiates all triggers. So, if your triggers have stopped running, please open the sheet with trigger refresh issues, and open API Connector. This will restore any triggers that have been disabled or deleted by Google.
  6. Google generally provides higher quotas for users with a paid Google Workspace business account. If you are using a standard (free) consumer account, upgrading to Google Workspace may help.
  7. Turn on request logging to better identify trigger issues.
Previous Print Timestamp
Next Set Data Destination

27 thoughts on “Scheduling”

    • Hi Mathias, the hourly update limit is because Google doesn’t let add-ons use triggers more than once an hour (documentation). If that ever changes, I’ll add in triggers for more frequent updates.

      Reply
  1. I will post my question here also.

    Is it possible to control the time of the schedule the triggers at a specific time within the hour or is it determined by Google?

    Reply
    • It’s determined by Google. They slightly randomize the time within the hour, so there is no way to force it to update at a specific time other than trial and error. If you haven’t already, you can add a timestamp to your request (located under Output Options) so that you know exactly when it executes. Once it runs the first time, Google will keep that time consistent day to day.

      Reply
    • Hi Jaco, sorry, there’s no way to manually hook into API Connector code. Even if you could, I’m not sure how manually adding the script in Google Sheets would help with running at the exact precise time every hour, since scheduling limits and execution times are set by Google, not API Connector. We already use Google’s programmatic triggers in the code.
      Please let me know if that answers your question, if not, I’ll be happy to explain further.

      Reply
  2. Would it be possible to set the output mode via the trigger creation/edit page?

    Right now I have to create multiple requests and then put them in the order they need to run in. If the I could set up one trigger to run a request in Overwrite and then, another to run it in Append I could do the same thing but with triggers instead of Requests. This would be a lot easier for me to manage since it’s just a run parameter that’s changing.

    Reply
    • Hey Fred, you are suggesting that the output mode gets set on the trigger screen rather than on the create report screen, right? That might be convenient in this specific case, but I would be concerned about splitting up where options get set, especially as a lot of people choose overwrite vs append mode without using scheduling at all. Also, it doesn’t seem too time consuming to click Actions > Copy and create a new version of a request with a different output mode, but maybe I’m missing something about your workflow.

      Let me know what you think.

      Reply
  3. First of all, thank you very much for creating this plugin.

    I use it to pull data from XML as a paid user. Everything works perfectly when I manually trigger it.

    However, when I use hourly schedule triggers, I constantly have problems. I can’t clearly tell when it is working or not. That’s why I often have to trigger manually.

    I guess you are sending these schedule triggers into a queue system, and it performs these operations in turn. In busy times, it doesn’t come to us.

    How can we overcome this problem?

    Reply
    • Hi Oguz, I think we already communicated about this via email but I’ll respond here as well. Thank you for the message and sorry you’re having trouble with your requests. Triggers are managed by Google, and the most likely cause of scheduling issues is that you’re running into Google’s 6 mins/execution scheduling limit. This article contains several suggestions for avoiding Google’s limitations (please check the section called “Troubleshooting Triggers”).

      Reply
  4. Hi!
    Yesterday, at 8.00 pm, schedule a trigger for the execution of the API request between 1.00 am. and 2.00 am.
    Today I consulted at 9.00am and it was never executed. What time system does the API Connector use? Thank you!

    Reply
    • Hi there, all triggers use the timezone of the spreadsheet. If your request didn’t complete when it should have, please turn on request logging and check the troubleshooting suggestions above. Just shoot me a message if you’d like help looking into it.

      Reply
  5. Hello. I’m a paying Mixed Analytics customer. I understand that Google scheduling cannot allow a user to specify when within an hour a job gets called (this is not a limitation of Mixed Analytics). I’m trying to get my task to run exactly on the hour (8:00:00, 9:00:00, etc.) 24 times a day, 365 days a year by setting up a macro in google sheets, and then scheduling the macro to run at exact times.

    My questions are – have you seen a way to schedule macros for an exact time? Can I call Mixed Analytics from within a google sheet macro? Is this a workaround to this problem or am I wasting my time?

    Reply
    • Sorry, I don’t know how to call our add-on from a macro, since Google only lets us access add-on functions through the menu, sidebar, custom functions, etc (the elements of the add-on itself). I tested creating a macro and it didn’t recognize any of my actions in the add-on.
      For something like this it might be better to create your own custom script (or have a developer write one) as Google’s time restrictions don’t apply to those.

      Reply
    • Hey John, sorry, no we haven’t. I think the problem is that there are many potential errors that wouldn’t be resolved by trying again (like authorization errors), and could even make it worse (like if the quota needs to reset). Scheduling only sends requests 1x max per hour, and we automatically insert 1.5 second delays between multi-query requests, so rate limits usually aren’t a problem. If by “YT” you meant YouTube, API Connector has a quota from YouTube that gets shared between all our users, so one possibility is that you hit our limit. We can contact Google to ask for a quota increase, and/or you can get your own API key so you pull from your own personal quota. Update: we received a quota increase from YouTube.

      Reply
  6. HI,
    Somehow the scheduling can only update the status daily/ weekly/ monthly. The option of “hourly” does not show up in the option. Is that true?

    Reply
    • Hourly scheduling is a Business plan feature, so if you don’t see it, you’re most likely not on that plan. To check your plan, please click the Account link in the main menu or footer of API Connector, or just send us a message via support.

      Reply
  7. I’m having an issue where the scheduled triggers that I run don’t really refresh the data, it just pastes the same data again. However, when I run the request manually by clicking ‘run’ on a specific request from the ‘requests’ tab, it updates the data properly. Any idea what might be causing this?

    Reply
    • That’s strange. I wonder if it could be a time or time-zone related issue, where the scheduled request pulls data for the same day, but when you pull it manually it’s now a new day such that it pulls fresh data. Could you please a) enable request logging and b) try setting your schedule to run a bit later? Or if you’d like to message support I can dig into this in more detail.

      Reply
  8. Tôi đã đặt lịch trình tự động kích hoạt chạy ” run ip ” khung giờ từ 2 – 3 am , nhưng tới nay là 2 ngày trôi qua nhưng ” run ip ” vẫn không tự động chạy theo đúng lịch , tôi phải chạy thủ công bằng thao tác bấm ” run ” . xin admin giải quyết vấn đề này giúp tôi . cảm ơn rất nhiều

    Reply
    • Can you please turn on request logging to see if there’s any error associated with these scheduled requests? I’ll also email you so we can look into this in more detail.

      Reply
  9. As a paying Mixed Analytics customer, we can schedule triggers. Over the years this has grown, some are no longer needed. How can we get a detailed overview of all the triggers? Via https://script.google.com/home/triggers I get to see 30+ daily requests but there is not much (no) detail in them, it would be nice to see through which Spreadsheet the trigger is running.

    Reply
    • You could navigate to Google Drive > Recent to see any sheets that have recently updated, but for a more comprehensive list please message support. We can query the backend server logs to get you a list of where your triggers are running. I’ll also add more accessible trigger metadata to our list for future development.

      Reply
    • You can’t create a button but you can do one of the following:
      1) Click Extensions > API Connector > Refresh All Now
      2) Use the IMPORTAPI custom function (see the section on fast cell-based refresh for info on refreshing requests via a checkbox)

      Reply

Leave a Reply to Harris Cancel reply

Table of Contents