V6.75 Upgrade 9-4-21

robert · September 5, 2021, 2:01am

Fixed the graphical automation history where the number of loop iterations displayed was one greater than the actual number of iterations performed. This only occurred when another action followed the loop. Only newly processed data going forward is fixed.
Refactored how the system searches for references within automations. Instead of using SQL “LIKE” queries for text within automation XML stored in MySQL, the system now uses Lucene. This greatly improves searching when the system has many automations. Functionality that is now much faster includes: assembly editor catalog searching for automations that use an assembly, renaming assemblies referenced in automations, the “referenced by” gear menu option in the automation editor’s dashboard, importing automations, the sending of “system” webhooks, and other server-side functionality.
Added a “Compare two arrays” action to the Transform Data app.
In the graphical automation history, if an action log is viewed for an action that wasn’t executed, a message appears instead of an error about the log not being found.
Fixed a regression where the automation execution engine was not processing trigger records for automations having a Date Time trigger in combination with one or more other triggers.
The automation editor and the graphical automation history now both display “AND” next to a Date Time trigger.
Fixed an issue where database connections would be leaked when auto-retries were being performed for 401 errors returned by OAuth v2 APIs.
When comparing versions of an assembly in the assembly editor, after the diagram loads an error about overlapping modules is no longer displayed.
When a linked automation having a webhook trigger is redeployed, the system no longer flags the webhook trigger(s) as being unconfigured.
When an automation having a webhook trigger is redeployed, the destination automation’s configured webhook URL that is displayed in the trigger settings is no longer overwritten with the webhook URL from the deployed automation.
If the system is configured to store logs in Amazon S3, when loading the log if an error is received from S3 the system now checks to see if the log can be loaded from the local filesystem. This handles cases where the system has not yet written logs to S3.
The Admin Console’s “Retries” screen now has a button to cancel all scheduled retries.
The system no longer automatically turns off automations if they encounter a missing app connection or receive a 401 unauthorized error from an API. The original impetus for this logic was to avoid the system running abandoned automations or automations that had no chance to successfully process. But this behavior was causing operational problems at scale, where many automations across accounts could be turned off due to API or system glitches. Now, an automation must specifically have its “halt automation on any alert” setting enabled for it to be automatically turned off due to a processing error.
The system now by default performs javascript string escaping for mapped fields in the System app’s “evaluate javascript expression” action and uses a new JSON parser introduced in the previous release. This functionality is controlled by hidden config flags and can be disabled if needed.
When deploying automations and the “show linked accounts” checkbox is checked, the system no longer returns accounts that only have deleted automations in the group of automations being deployed.
When clicking a filtered data row in the automation history drilldown, the graphical history is now displayed instead of only a dialog with the filter information, so that the trigger’s log can be inspected.
In the assembly editor, when searching for automations that use apps, triggers, and actions the search results are now ordered by automations that are turned on, then automation id numbers in descending order. This puts automations that are turned on at the top of the list, with the list also sorted by newest automations first.
The Math Operation module has been deprecated and replaced with a copy whose only change is to specify the number of rounding decimal places in the body of the module instead of as a setting. This allows the number of decimal places for rounding to be wired into the module.
Added a “round” action to the Transform Number app that rounds numbers to the specified number of decimal places.
The Admin Console’s Work Queue screen now has an option to disable one or more apps. This allows all automations using the app(s) to not be processed by the work queue.
In the automation history screen, if an automation was queued but never processed by the system old entries in the history view now show a “queued but not executed” information entry instead of “queued for execution” which made it seem it was still queued for processing.
Have added a “configure UI” action to the Human Interaction app that can be used to configure UI elements in the moderation queue. Each scope per automation can have its own configuration. This action can only be used once in an automation.
Added a “Find value in array” action to the Transform Data app. Returns the total number of matches of a value in an array.
The system now has functionality to snooze an automation for an extended period of time – days, months, years. The Date Time app now has a new “snooze automation” action. It allows a date and time to be specified in the same manner as the Transform Date Time app, which includes natural language such as “in two days” or “next Friday at noon”. When the action is executed with a configured datetime in the future, the automation’s execution is suspended and the automation exits the work queue. The automation will continue at the next action after the snooze action either when the configured datetime is reached or when the snooze is manually cancelled in the automation’s history. All snoozed transactions can be manually woken up from the automation history at once via a single button, by choosing to filter for only snoozed transactions. If the system is rebooted, any snoozed transactions that should have been executed while the system was offline are immediately placed into the work queue for processing while snoozed transactions scheduled to wake up in the future are placed into the scheduler.
The System app has a new “serialize automation execution” action. The purpose of the action is to only allow one automation at a time to execute within a defined group and scope. The group is specified by a name. The scope can be either a single account or any account within a linked account group. The action should be added as the first step of an automation. The main usage of the new action is when automations are invoked via the “execute automation” action, especially when “wait to complete” is turned off, in order to avoid API rate limits caused by multiple automations running in parallel that all use the same connected API account.
The list of executing automations in the Admin Console’s work queue screen is now sorted in the order that automations began executing, rather than the list being randomly sorted (a side effect of Java data structure iteration). This can make it easier to monitor a particular automation’s processing in a busy system. Also, the longest-running automations will now always be at the top of the list.
The Date Time “pause for a specified number of seconds” action can now accept fractional seconds, e.g. 0.5 for 1/2 a second. When using it in a loop to slow down API calls a value of 1 second is typically too slow. Generally a value of 0.5 or 0.25 should be enough for most APIs to avoid rate limit errors.
The System app’s “execute automation” action now has input fields to define a latch group and account scope, similar to the serialized execution group. With a latch group, a parent automation can execute multiple child automations all in the same group, with “wait to complete” set to false. Then the new System app “wait for latched automations” action can be used by the parent automation to wait for all the forked child automations running in parallel to complete their processing before performing additional steps.
Accounts having the Assembly Developer or Switch Account permissions will now see a “Latch group…” gear menu option in the automation editor’s dashboard. This allows the newly added latch functionality to be utilized for automations invoked by polling or instant triggers. The latch group assigned via the “execute automation” action takes precedence over the latch group assigned via the gear menu.
When automations are exported & imported or deployed, their attributes such as their concurrency limit and the newly added latch group are now copied.
The account settings menu in the automation editor has a new setting to control the default error handling for actions added to automations, which can be either to halt on errors or to continue on non-critical errors.
If an automation having an instant trigger that requires a unique URL is published to another server (typically a production server in a dev-prod arrangement), any instant triggers requiring unique URL’s then become locked and cannot be removed from the automation in the source system (typically the development system). This is done to avoid the scenario where live automations in the production system would stop processing received webhook payloads if the development automation was later edited and the instant trigger was removed and replaced.
After the assembly editor catalog is refreshed, the vertical scrolling no longer jumps around as the catalog is scrolled.
Pressing the Escape key will now close the topmost dialog window, unless the dialog contains a text editor and the text editor has focus.
Fixed the “Download log” button in the automation editor’s graphical history so that the log will actually be downloaded instead of a pop-up window opening and closing without the log being downloaded.
The automation task history screen headers now show the countdown timer until the next execution for polling automations that are turned on. Automations that only have instant triggers or ones that are turned off show a corresponding message.
If an automation’s name starts with “ADMIN” (case sensitive) and the signed-in account does not have the “Switch Account” permission, the automation will not display its on/off switch or its gear menu in the dashboard.
When the automation editor’s dashboard is initially loaded, inactive automations are now sorted by name per the default ordering.
When deploying automations, the deployment dialog now displays the total number of selected accounts in its title bar and in the action button text.
Added a “dump items from bucket” action to the system’s Collector app, which will output the items in the bucket as a single array element.
The automation editor will now display an alert if more than one account is editing the same automation at the same time. This can happen when multiple system admins have switched to the same account.
After an edit has been made to an automation, a “keep for later” button now appears in the screen’s header. This button is equivalent to an immediate auto-save and the closing of the editor, allowing the edit session to be paused and resumed later.
The assembly editor’s “File” menu at the top right now has a “keep for later” menu option. This option is equivalent to an immediate auto-save and the closing of the assembly tab, allowing the edit session to be paused and resumed later.
When editing automation field mappings, clicking the “eye” icon in a mapped field now opens the list of mappable fields and filters the list to show the mapped action, scrolling it into view if necessary. Double-clicking the eye icon now performs the previous functionality of opening the automation diagram and highlighting the mapped action.
Fixed an issue where the system could end up deleting automation log files on the filesystem and in Amazon S3 before they should have been pruned according to their age.
Fixed an issue where the “execute automation” action, when configured to wait until completion, would time out according to the system’s HTTP socket timeout setting instead of waiting as long as needed for the automation to complete.
When the system displays a duration of time, it is now broken down into days, hours, minutes, and seconds (e.g. 1d 2h 3min 4sec 5ms) rather than being displayed in milliseconds.
When a search is performed in the automation editor for action or field mappings, if any results are found a new entry field for entering replacement text is displayed. Entering replacement text will replace all exact matches (case sensitive) of either action names or mapped field names, depending on the selected search option.
Fixed an issue with the automation execution engine where a disabled action as the last step of a conditional branch nested within an outer loop would cause an infinite loop.
Fixed conditionals in the automation editor such that their name text won’t wrap past their bottom branch flow lines.