Working with Safeguards in Integrator

One of the most frequently asked questions administrators have about Integrator is what to do when they get a notification saying their job is on hold or see SAFEGUARD ACTIVATED on the jobs list.

What is a Safeguard?

A message like the one above could be intimidating! But don’t worry, it’s a good thing. You know Integrator has been hard at work looking at every data change and it’s giving you the opportunity to accept or reject those changes before they propagate to other systems. This is one of the ways in which Integrator helps you maintain data quality in your organization.

Safeguards protect your apps from bad data which may be caused by user or system error. They are a key feature built into Integrator and available for every agent and job. For example, let’s say a typo in your SIS caused a large group of students at one school to have their schedules altered. The safeguard would likely catch this before it impacted your downstream systems like Active Directory or your LMS, etc.

In the most simple terms, Safeguard analyzes data changes and “trips” like a circuit breaker when too many changes are detected. The “trip” occurs while Integrator is performing source system change data capture and it will halt a job before the data is loaded into a target system, placing the agent in “inactive” status. Until some action is taken by a user, the agent will not run. Under certain conditions, especially if thresholds are set too low or during heavy data change periods like back to school, registration, and year-end transition, the safeguard can trip often.

What happens when the safeguard “trips”?

When the safeguard for a particular job trips the job will be set to “Inactive” and you’ll get an email notification which will tell you the job is “On hold” or the “Safeguard is activated” depending on your configuration. Any scheduled job Integrator attempts after that will fail and you’ll get an email notification which will tell you something like “No extract, agent is inactive. No load, agent is inactive.” depending on your configuration. The job won’t run again until someone takes action.

What do I do if I get an email saying my agent is inactive or on safeguard hold?

1. Login to your Integrator portal at https://dis.readwritedigital.com/. You will see your most recent jobs.

2. In the “Job Status” column you are looking for “SAFEGUARD ACTIVATED” like this:

3. If an agent has parent / child agents (multiple agents under a parent) and you see “SAFEGUARD ACTIVATED” on the parent row, expand the parent row and look for that message on a child agent. It may look like this:

4. This gives you a quick idea of why the safeguard tripped.

5. To see the detail, click the SAFEGUARD ACTIVATED link. You will then be looking at the detailed job data. Click the “Change Data” tab to see the actual changes. You can filter any of the data in the grid and see exactly what data fields changed. For update records, you can hover over any red data fields to see what the original data was prior to the change. For more information on how to analyze job data, see this post.

6. Once you have completed your analysis of the changes, click on the “Jobs” top-level navigation link to get back to the Jobs page.

7. To take action on the agent, check the checkbox next to the agent name so it looks like this:

NOTE: If you are taking action on an agent with children, you generally always want to take action at the parent level like shown with the iStation parent above. You will see the agent information indicates the parent is on hold at the ISIPER and ISPAR child agents. Only take action on specific children if you have a specific reason to do so and understand the potential side-effects of doing so.

Also, it’s not necessary to select a parent and it’s children. Just select the parent agent.

8. With the proper agent (or agents) selected, select the action you want to take from the popup menu below the job list:

Essentially, if the data looks fine, select the “Run load…” option otherwise select the “Rollback…” option. For a job on safeguard hold, those are the only two options that will work.

9. Click the “Execute” button and click “Ok” in the confirmation dialog that appears. The jobs will now run and once completed, will be ready to execute on their next scheduled run.

How do I control the way safeguards work?

You may be getting safeguards too often, meaning the safeguard is too sensitive or you may not be seeing many or any, meaning the safeguard is not really guarding against potential problems. The good news is you can control everything about safeguards. You do this under “Agents”

1. Click “Agents” in the top-level navigation.

2. In the list, find the agent you want to work with. Typically you will want to set safeguards at the child agent level, not a parent agent. On the agent list, a parent won’t have a parent listed in the “Parent Agent” column but a child will look like the “College Board” and “Score Files” agents in the screenshot below.

3. Click the agent you want to configure and scroll down a bit to see the following section where you can turn the safeguard on or off and set it’s thresholds:

4. If you have an agent that runs multiple times a day, the maximum daily change threshold should reflect the cumulative number of changes you would expect to see during the entire day. To get a sense of this you can do some simple math with the changes you are seeing for the job every day and you can also examine the change graph for the job at the top of the Jobs page. Click the chevron at the top right to expose the graph.

The “Month” period gives a good indicator of activity over time and you can remove “outlier” jobs by clicking the “Hide outlier data” checkbox at the top of the chart. Here we see 12 changes are the max “normal” changes this school has seen in the month so far.