Partial Status

When an Evaluation is executed in Alloy, an overall status and corresponding code is generated. The status serves as an indicator of the overall success of an evaluation completing as it progresses through the Workflow. If the evaluation is unable to complete, the default behavior will result in an early exit from the Workflow with a partial result. This article will cover in depth what partial results are and how to review them.

This article will cover the following sections:

• Partial Results
• Understanding a Partial Result
• Partial Information in Raw Response
• Troubleshooting Partial Results

Partial Results

Partial results indicate that an Evaluation could not fully complete running through the Workflow due to missing information.

A Workflow expects certain information to always be present in the API response based on the configured attributes. Workflow attributes are specific data points that are used for decision-making or informational purposes. They can be derived from various sources such as Alloy's API or third-party services and can be used to create tags, set conditions, or influence the outcome of a workflow.

If the data vendor API response lacks information the Workflow expects for a given attribute—whether the information is missing or there was a data service failure—then the Workflow will be unable to determine if the thresholds have been met to set a tag, run a condition, or influence an outcome. If the missing data occurs, the Workflow will evaluate how to handle the error based on the configured service service failure handling. The default service failure handling is set to “Exit Workflow” when data is missing.

Evaluations that have a partial result and exit the Workflow early will have an error indicator that can be viewed from the Evaluation or Application Queue. The evaluation will display in the raw API response with the status code 206. The returned raw response will include details of the error within a warnings object.

In the dashboard, the reason for the partial status will also be displayed in a banner on the Evaluation, with suggestions on how to resolve the error. An evaluation can be viewed either from the Evaluation Summary page or when reviewing it as part of an overall Application.

Understanding a Partial Result

The reason for why an evaluation returned a partial error is suggested within the banner displayed on the evaluation or application details page.

Sample partial banner

There are two main reasons why an evaluation may return a partial error:

• The tag could not be decided
  • The service that the tag depends on attributes from was down at the time
  • The service returned a response but was missing the fields that this tag looks at
• The service failed to run; this could then be due to either of these reasons:
  • User error (the workflow is misconfigured; for example, the credentials are incorrect)
  • The service was down at the time

The partial banner can reflect both of these types of errors and multiple errors for each type.

The tag could not be decided

This error is is returned when the Workflow could not determine if a tag should be set due to missing information from the vendor API response. This error can be returned when a data service ran without errors but information is missing, or if it failed.

This type of partial reason will be reflected in the format of: The tag TAG_NAME could not be decided because SERVICE_NAME didn't return ATTRIBUTE_NAME where TAG_NAME is a placeholder for the tag in your workflow, SERVICE_NAME is a placeholder for which service did not return information, and ATTRIBUTE_NAME is a placeholder for which attribute didn’t return the data the workflow is requiring.

This partial error can also be returned in the situation where a data service completed running or had an error. A data service may not always return the data from their end, even after successfully running. If a data service failed, which is reflected by a ❌ next to the data vendor’s name in the Data Services table, any tags dependent on the data service will have an error.

How to resolve

In either case, the Workflow rules can be updated with different service failure handling on the Tag(s) in question, or additional tags can be included upon failure to give more insight on the Evaluation Summary. Once this is done, the evaluation can be rerun.

The service failed to run

This error is returned either when a service has failed or when there has been a user error. A data service that failed to run is reflected by a ❌ next to the data vendor’s name in the Data Services table.

This type of partial reason will be reflected in the format of The service "SERVICE_NAME" failed to run where the SERVICE_NAME is a placeholder for which service failed to run.

How to resolve

Since there are two possible reasons for a service failing to run, we recommend going to the Data Sources section and clicking on the service(s) with the ❌ to see what the Error message says. If the Error message relates to credentials or configuration, address this error before re-running.

In the following example, the Socure service has failed. Clicking into the service opens the raw response modal. From the Compiled Response view, you can see the API response the vendor returned, which also may contain the reason for the error. In this example, the error is Invalid email. Please resubmit the transaction using an email address conforming to RFC 5322 and the action would be to re-run the application with a corrected email address.

Sample data vendor "Socure" displays "Failed" icon in the "Data Sources" section of an Evaluation Summary

Sample data vendor response with error

Otherwise, assume this was caused by the service being down at the time of the evaluation, and then try re-running. Data services that were unavailable may also sometimes reflect error messages such as Internal Server Error when clicking on the service to review the error.

Partial Information in Raw Response

The Raw Response for an Evaluation will contain the reason for the partials, allowing your team to programmatically determine the cause.

Example of warnings object:

"warnings": {
    "blocked_actions": [
      {
        "blocked": {
          "action_name": "IDV Pass",
          "action_type": "tag"
        },
        "blocking": {
          "action_name": "Onfido",
          "attribute_name": "Check Result"
        }
      }
    ]
  }

Troubleshooting a Partial Result

Why did my application exit early?

Workflows exit early if a tag expects data from a service that is missing, often due to a data service error. This behavior is default for tags, but you can change the setting under “Service Failure Handling” in the tags options.

Why did the application return a certain outcome?

Even with partial results, evaluations still have an attached outcome. The Workflow uses whatever data is available to decide an outcome based on configured rules and tags, setting the default outcome if no decision can be made. This default outcome is the “Else” section in your outcome logic.

While some data may be unavailable, there will still be services that executed successfully and tags that were set without dependence on that data. While the partial result indicates that the Workflow ran partially and there is incomplete information on the evaluation, the Workflow will still attempt to determine an outcome based on the configured outcome rules and the established tags.

In such instances, the Workflow uses what is available to decide on an outcome based upon the rules set in the workflow.

If no decision can be made, then the default outcome will be set. This is the “Else” section in your outcome logic.