Manually edit database log

Is there a way to edit the database dump to remove a unique ID so just that one can be ran again?

Hi Brent,

Sorry, there’s no currently no functionality for that in the UI.

ok. what about a way to force a retry of a record?

Basically there were some items that were processed and went through. Then changes were made and they were deleted and need to be processed again.

I can remove them manually in the database. Will need to know which system, which automation, and which unique ID’s.

automation: 82d4cec93e714a598d1375b9293b92fd


Basically everything after 2019-06-21 19:37:12.0

Is there any plans to opening up access to our own database to handle unique identifiers ourselves? I could see that being extemely useful.

I have modified those records in the database so they can be reprocessed. I can’t delete them manually very easily b/c they are joined across tables, so I just appended “X” to each value which will have the same effect.

The next system release has a new screen for managing two-way sync database entries:

Will see about building a similar screen for other database entries like the unique ids stored by polling automations.

that would be great so i wont have to bother you next time I need to reprocess something

The ability to edit polling automation data is now included in the next upcoming release:

Awesome. that will be massively useful.

When is the next release scheduled to go to production?

Also, could you edit one more for me? I missed one yesterday.
Same automation: 82d4cec93e714a598d1375b9293b92fd
ID: 00041927

I have done the same change for 00041927, sorry for the delay. The forum is not good at notifying quickly about posts via email. I have tried to adjust its settings to no avail yet.

The new release is a week or two away. Here is the current list of changes in it:

  1. The performance of the appRPC handleReceivedUnaryProtocolPayload() method, used by Unary Protocol Threads to invoke automations, has been greatly improved by optimizing a SQL query to utilize an existing index.

  2. When dragging items in the assembly editor, if a modal opens the drag is now halted. Previously the dragging would not stop, making it impossible to click in the modal.

  3. The automation name entry field at the top of the automation builder screen is now limited to 100 characters.

  4. The Transform Date Time “modify and reformat date time” action now supports a timezone offset value of “ACCOUNT_UTC”, which converts inputs in the account’s timezone to GMT.

  5. When an action in an automation is configured to “continue if error”, an error is no longer written into the automation’s task history log.

  6. Automation names in the automation editor’s dashboard are now fixed width, to avoid the screen jumping around as polling countdown timers are updated.

  7. Fixed a null pointer exception that would occur in the server runtime for the Trigger Two-Way Sync module.

  8. When sharing items in the assembly editor, the share settings dialog now has a checkbox to show only accounts the item is currently shared with.

  9. Accounts having the “Automation Templates” permission now see template annotation entry fields underneath all trigger and action settings when editing automations in the automation editor. These annotations will appear when template automations are initially configured by end users who install the templates and turn them on for the first time. The annotation text can contain HTML markup, for embedding links/images/videos/etc.

  10. The search entry field in the automation editor’s dashboard now requires pressing ENTER to search. An onscreen prompt indicates this. A “Waiting…” spinner now appears when searching, b/c searching in dashboards with lots of automations can take a few seconds to complete.

  11. When saving automations in the automation editor, a “Validating…” spinner now appears briefly as the system validates the automation’s field mappings. This can take a few seconds for automations with lots of actions and fields.

  12. Added a “Conditional Execution” module that is a hybrid of the existing [Single Value] and [Data Stream] variations. This module allows two scalar values to be tested, while allowing the nested item to emit either a scalar value or a data stream. It fills a gap where the existing [Single Value] module cannot emit a data stream and the existing [Data Stream] module cannot easily test two scalar values without having to place the left side value into a data stream first. An expected common usage scenario is to conditionally execute the nested item based on the output from the When In module.

  13. The “Get dynamic action output fields” subassembly has been deprecated and replaced with module “Action - Extract Output Data Fields” that provides equivalent functionality but with maximum runtime performance by using native Java on the server.

  14. The “Show only required fields” and “Show only mapped fields” checkbox functionality in the automation editor’s action settings has been optimized so it completes faster by minimizing rendering of the fields as they are hidden or shown.

  15. Internal changes have been made to help reduce the system’s database storage for automation history going forward. Existing data already stored in the database is not modified.

  16. In the automation editor, when clicking on action data fields the popover of mappable data fields now renders the popover immediately and then renders the list of fields separately, so that the popover can appear as soon as a field is clicked rather than being delayed to wait for all fields to render first.

  17. Text entry field borders no longer initialize to black and then quickly transition to gray. Instead, they initialize to gray.

  18. Action parameters can now be optionally data-mapped. When editing an action in the automation editor, data-mappable parameters will have a “Data map?” checkbox next to them. When checked, the parameter value can be mapped to a field or hardcoded by hand. The data-mapped value can either be the item’s name or the item’s id. In the case of populated dropdowns, the action will fetch all the values of the dropdown when the automation runs and compare the contents to the data-mapped value. The first item with a name or id matching the data-mapped value will be used. Data-mapped parameters are not currently available for actions that have dynamic input fields or dynamic output fields. When a template is installed and configured, any data-mapped parameters are not shown in the initial configuration dialog when the installed template is first turned on.

  19. The Trigger Two-Way Sync module will no longer emit an item for processing if its id is in the database and if its created timestamp equals its last modified timestamp.

  20. The Quickbase “add row” and “update row” actions now add/update data values in a manner that no longer causes surrounding whitespace characters.

  21. Fixed an issue where automations would not by default not send error alert emails to their owners.

  22. Automation alert emails sent to both the system admin and to the owners of automations now have a link at the bottom to suppress email alerts from the automation for errors starting with the same error message.

  23. Automation alert emails sent to the system admin now have a link at the bottom to stop receiving all email alerts for the automation, plus a link to stop receiving all email alerts for all automations in the account (newly created automations in the account may still send error alerts, however).

  24. The polling schedule and daily execution window for polling automations are now retained when polling automations are exported and imported, deployed, or created from templates.

  25. Accounts having the “Assembly Developer” or “Switch Account” permissions will see a new automation gear menu option under the “Processing” menu for two-way sync automations named “Two-way sync mappings” that opens a new screen for managing two-way sync mappings in the database. Mappings can be searched, modified, and deleted.

  26. The default automation email alert delivery frequency is now daily instead of immediately. The setting can be changed for each account from the automation editor’s Settings menu under the account at the top right.

  27. The Salesforce “Two-way sync add or update object record” action now supports matching logic.

  28. When wires are joined to modules in the assembly editor, if the assembly execution engine needs to be run a progress dialog now appears instead of just a message in green at the top of the screen.

  29. The system no longer stores processed data rows in the database due to causing large database storage and slow search performance. Now, the system stores processed data rows (if the system and automation are configured to store processed data) on the filesystem using the Lucene library. This change is expected to greatly reduce database storage and improve automation history search performance. Existing data rows currently in the database will eventually be pruned based on the system’s configured setting for automation history log age.

  30. The Subassembly Output (Sync Action) module now only emits the selected data stream rather than all input data streams, so that the Action Two-Way Sync module works as expected without reporting a runtime error that expected information cannot be found.

  31. Fixed the Trigger Two-Way Sync module so it won’t hang the automation when processing the first sync.

  32. The Trigger Two-Way Sync module has been deprecated and replaced with a new version that supports enhanced two-way sync logic where items updated in one system are linked to a record in the other system on-the-fly by using matching logic in the create actions. The new module no longer uses created timestamps or has the option to determine when items are created based on equal timestamps or the item not being in the database.

  33. Accounts having the “Assembly Developer” or “Switch Account” permissions will see a new automation gear menu option under the “Processing” menu for polling automations named “Manage stored data” that opens a new screen for managing polling data in the database. This will primarily consist of data row unique ids for successfully processed data rows, but may also include other data stored by the automation. Entries can be searched, modified, and deleted.

  34. Fixed issues throughout the system in the way it converts UTC timestamps stored in the database to an account’s local timezone. This primarily impacts automation task history, such that historical records will now show incorrect timestamps but new records going forward will be correct.

Looks like there might be a bit of a bug in the update.

Automation 9315c2efb9de4f9880298ba9a2cb3f31

I ran a test and process 3 rows. Ran through. Great. Then I deleted them to try running them again.
Went in great. no issue yet.

Turn the automation back on and try running it normally, trigger returns zero rows (even though I know there are rows to process and have verified).

Do some checking and try running in test again. it tries to run the same 3 rows that already processed.

That seems odd.

So I look into the debug log for the instance where it ran and pulled nothing. I can see rows on there like so where the emit new items module says it exists like so…
Sun Jul 07 21:25:10.203 EDT 2019 – Assembly [Trigger #1 (Salesforce - Modernistic Troy - New Project)] Module [Trigger - Emit New Items #6] Not processing existing identifier = [00068140]

now, 00068140 never ran. I verified it never made it to to final system either. the row was never processed but it shows up on the list.

So I looked in the database log and 00068140 does not exist there either. BUT I do see it if I search in manage data.

So, I deleted that row from the manage data screen. ran the automation in normal and it processed that one row.

I think something happened where when i ran it in test the first time it saved all the data it pulled as processed even though it only ran the 3 since it was in test.

I did spot check a few more values that did not run and it is the same. they exist in the manage data screen but not in the database log.