Question about the emit new items functionality.
I know the module stores whatever unique key you assign to it to a database. At what point does it store it? At the end of the trigger? At the end of the automation?
What happens if the automation fails on the action side? Will the trigger still be able to pull those items again or are they stored in the db and filtered out?
Just trying to get an idea of how it works.
Thanks
For polling automations, the data row’s id is stored into the database either:
or
As each action executes the execution state is saved into the database. So if a failure occurs in an action the data row can be later retried from where it left off.
Depending on the system’s configuration it will automatically retry certain errors in 5, then 10, then 15 minutes. Things like communication errors or 500 errors from the API that are expected to be transient where waiting to retry might result in success.
So generally the Trigger - Emit New Items module will never emit the same row id again. The data row is stored in the database until it either successfully processes (either automatically by the system or manually by the user) or until the configured age limit of automation history storage is reached, e.g. 30 days.
If an automation’s Data Storage option is disabled then the data row is never stored in the database. If it fails it cannot be retried:
Thanks.
And on the initial run? How does it know to not run everything existing and only pull new ones?
I would assume upon initial launch it pulls and stores then references the db from that point forward to only grab legitimately new items.
Correct, if the Trigger - Emit New Items modules finds zero ids in the database then it assumes it is being run for the first time and all row ids are stored. If no row ids exist, a dummy value is stored. Then as data rows are fetched row ids not in the database are processed.
ok. last question (i think).
Hypothetically, if I wanted to purge the database and start fresh, is there a way to do so?
Good questions!
Polling automations have this Reset menu option that removes all stored data row ids from the database:
perfect. so in theory if I had an automation running and needed to load a bunch of historical transactions, I could reset the automation, load them, then when it runs it will store everything in the db and ignore it all from future pulls.
learn something new every day.
Thanks
Bulk loading is an altogether different thing than a polling automation.
See the section on “export triggers” starting page 115 here:
sorry. what i meant was if I needed to load a bunch of historical into my source system (the one the trigger is polling) and I didnt want those to process. I could load them and clear the history so the next time it runs it would ignore everything and start fresh.
Correct, you can load records, reset the automation, then it will start fresh.
