REST Connector
Definition
The REST Connector aimed at provisioning remote HTTP applications.
It is based on the Groovy HTTP Client. The reader is supposed familiar with its usage.
The Groovy HTTP Client enables to craft (and chain if needed) HTTP requests in a flexible way, meeting customer’s specific requirements.
Here is an example of a remote user creation, the user is then added to groups:
// The user to create. The Groovy binding "ACCOUNT" contains the IDM object attributes mapped to the user's attributes
def user = [
uid: ACCOUNT.uid,
givenName: ACCOUNT.givenName,
surname: ACCOUNT.surname,
email: ACCOUNT.email,
birthDate: ACCOUNT.birthDate
]
// The remote application expects the user's attributes to be located under the root "data" JSON key
REST.post('/') { json([data : user]) }
// Add the user to groups
ACCOUNT.groups?.each {
REST.post('/' + user.uid + '/~groups/' + it)
}
// By convention the user's unique id must be returned upon the user creation
ACCOUNT.uid
Configuration
You can access the REST Connector configuration :
by clicking on "Synchronization" → "REST Connectors"
by clicking on "System" → "Configurations"->”Synchronization Service” and perform an import/export.
General Configuration Principles
A Connector is usually configured using keys and values, for example:
<connectorProperty>
<name>filePath</name>
<value>/csv/test.csv</value>
</connectorProperty>
This works well for simple key/value use cases, such as the CSV Connector.
The Generic REST Connector configuration is however far more complex, the above key/value model is not adapted to its complexity.
The principles are thus:
configure the REST Connector using a specialized
RestConnectorDefinition
, which is a regular configuration aggregate, just likeCorrelationDefinition
for exampleconfigure as usual a
ConnectorDefinition
which holds a single propertyrestConnectorDefinitionId
whose value references the id of the aboveRestConnectorDefinition

Properties
The following table lists the configuration properties of RestConnectorDefinition
.
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
connectorType |
| NO | The possible connector types, useful to parse a response (JSONPath vs XPath) | JSON, XML |
connectionDefinition |
| YES | The connection settings, such as TCP timeout, HTTP proxy, etc. | - |
authenticationDefinition |
| YES | Authentication settings, see dedicated section. | - |
operationsDefinition |
| YES | The configuration of each Connector operation (get, create, patch, etc.) | - |
responseParsingDefinition |
| YES | The response parsing settings. | - |
Connection Definition
The connection settings, such as TCP timeout, HTTP proxy, etc
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
connectTimeoutMs |
| NO | Determines the timeout in milliseconds until a connection is established. A timeout value of zero is interpreted as an infinite timeout. A negative value is interpreted as undefined (system default). | Between 1 and 5000. Default: 5000 |
socketTimeoutMs |
| NO | Defines the socket timeout in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets. | Between 1 and 10 000. Default: 5000 |
connectionRequestTimeoutMs |
| NO | Returns the timeout in milliseconds used when requesting a connection from the connection manager. A timeout value of zero is interpreted as an infinite timeout. A negative value is interpreted as undefined (system default). | Between 1 and 5000. Default: 5000 |
bypassTlsValidation |
| NO | If true the validity of the application's server TLS certificate won't be checked. | true, false |
tlsProtocol |
| NO | The TLS protocol to use. | TLS TLSv1.1 TLSv1.2 TLSv1.3 |
disablePooling |
| NO | Whether connection pooling is disabled. If disabled, a new TCP connection is created for each request. | true, false |
maxTotal |
| NO | Maximum size of the connection pool. | Between 1 and 10. Default: 10 |
maxPerRoute |
| NO | Maximum size of the connection pool for a given route (protocol / host /port) | Between 1 and 10. Default: 10 |
connectionRequestTimeoutMs |
| NO | The timeout used when requesting a connection from the pool. 0 means an infinite timeout. | Between 1 and 5000. Default: 5000. |
timeToLiveMs |
| NO | The maximum time to live of pooled connections. | Between 0 and 3600 000. Default: 900 000 (15 minutes) |
keepAlive |
| NO | Whether to send a periodical "keep alive" probe to the peer. The frequency is system-dependent. | true, false |
Operations Definition
Parent container of the Connector operations configurations
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
defaultUrlPrefix |
| NO | A default URL prefix common to all Connector methods. It is automatically prepended to method URLs defined in Groovy scripts if they do not start with | - |
defaultContentTypeHeader |
| NO | A default | application/json;charset=UTF-8 |
defaultAcceptHeader |
| NO | A default | */* |
objectUniqueIdentifier |
| YES | the name of the attribute containing the application object's unique identifier, such as "uid". | - |
operationDefinitions |
| YES | The configuration of each Connector operation (get user, create user, etc.) | - |
Operation Definition
Configuration of a single Connector operation such as get user, create user, etc.
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
operation |
| YES | The name of the Connector operation | GET_OBJECT, CREATE_OBJECT, PATCH_OBJECT, DELETE_OBJECT, SEARCH_OBJECTS, DISCOVER_OBJECTS |
action |
| YES | The Groovy Script as a ComputeRule implementing the Connector operation, which may include several chained HTTP methods calls | - |
Response Parsing Definition
Container of response parsers, see ResponseParserDefinition
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
responseParserDefinitions |
| YES | This list of available response parsers. If the "parse(parserName)" method is not explicitly called on a Response, such as | - |
Response Parser Definition
How to parse a HTTP response body.
Property Name | Type | Mandatory | Description | Values (default value in bold) |
---|---|---|---|---|
parserName |
| YES | The parser name, that can be referenced when explicitly calling the "parse" method on a Response, such as | - |
objectPath |
| NO | The JSON path or XML path expression pointing to the object embedded in the application's response. If null, it is supposed that the object is at the root of the response. For a list of objects returned by a "search" operation, it points to the list containing the objects. | null |
attributesPaths |
| YES | How to extract the object's attribute values from an application's response. The
| - |
Authentication Configuration
TODO
Impersonating the End User Identity with the Google REST Connector
The Google REST Connector authenticates to the Google platform with a JWT OAuth2 profile, see href="https://tools.ietf.org/html/rfc7523
A signed JWT token is thus embedded in each REST request sent to Google. A “subject” is part of the JWT token, identifying the account at the origin of the REST operation.
Most provisioning operations are performed with a single technical JWT subject configured globally in the REST Connector’s settings. However, to access Google's delegate APIs (for example to perform an “email transfer” operation), some requests must be performed on behalf of the end-user targeted by the operation. The JWT subject must thus be set with the end-user’s identity, not with the technical account’s.
To achieve this, a method authSubject
is exposed in the API enabling to build REST requests:
/**
* Force the subject of an authentication request, overriding global settings of the REST Connector's authentication
* configuration.
* <p>
* This is namely useful to access Google's delegate APIs, where requests must be performed on behalf of a subject.
*
* @param subject set in the authentication request sent to the remote REST application
*/
RestConnectorRequestBuilder authSubject(String subject);
For example, to get a specific delegate, provided that the delegator’s email is available in the ACCOUNT.email
variable:
REST.get('https://gmail.googleapis.com/gmail/v1/users/' + ACCOUNT.email + '/settings/delegates/{delegateEmail}') { authSubject(ACCOUNT.email) }
Example
Here is an example of XML configuration.
The URL to access the configuration is: /{tenant}/api/sync/conf/rest-connector-definitions
.
The corresponding ConnectorDefinition
referencing the above RestConnectorDefinition
is represented below.
Writing Groovy Scripts
6 connector operations need to be configured as Groovy scripts:
get a remote account
search remote accounts
discover remote accounts (optional, only needed if a “discovery task” is executed)
create a remote account
patch a remote account
delete a remote account
Those 6 operations correspond to the configuration of 6 OperationDefinition
items (see above).
Groovy Contexts
Groovy scripts access external values through contexts (the technical term is "bindings") that are injected upon execution.
Those contexts are detailed below:
Context Variable Name | Java Type | Description | Available For Connector Operation | Example Usage |
---|---|---|---|---|
| GroovyRestClient | The REST client on which HTTP methods can be called, such as GET, POST, PUT, etc. | All operations | HTTP.post('https://...') |
| String | The identifier of the object to process on the remote application, not to be confused with the Memority IDM id | GET | def id = OBJECT_ID |
| Account | The object to process on the remote application, whose attributes are the target application's attributes (not IDM attributes) | CREATE | ACCOUNT.firstName |
| ApiObject | The IDM object, source of provisioning | CREATE | IDM_OBJECT.firstName |
| List<AttributeDelta> | Pertains to a Patch operation only, the attribute patches to be applied on the remote application object | PATCH | PATCH.getValuesToAdd() |
| Map<String, Object> | The search criteria provided by | SEARCH | SEARCH_PARAMS.login |
| PageRequest | Paging information enabling to keep track of the paged search state between consecutive search iterations. | SEARCH | def idx = PAGE.index def offset = PAGE.offset def token = PAGE.pagingToken as String |
| Instant | The last time a sync task or prov task was launched. For the inbound case, this information may be useful for the REST connector to perform a "find all" search operation filtering on "last updated since". | SEARCH | def lastExecDate = LAST_EXEC_DATE as Instant |
| ActivationSituation (enum) | May be null. A non-null value indicates one of the 3 possible situations:
| PATCH | if (ACTIVATION_SITUATION == IDM_OBJECT_DELETED) { … do something… } |
| PasswordGeneratorProvider | Enable to generate a random password. | All operations | PASSWORD.length-16).generate() |
| Map | External context | PATCH in some situations | def someBusinessProp = EXTERNAL.someBusinessProp as String |
Connector Operations
This section lists, for each connector operation, the expected output of the Groovy script implementing the operation.
Connector Operation | Description | Expected Groovy Script Output |
---|---|---|
| Create a new account on a remote application. The account's identifier is expected to be returned. |
|
| Update an account on a remote application. No return value is expected. |
|
| Delete an account from a remote application. No return value is expected. |
|
| Fetch a single account by unique id on a remote application. This is generally more optimal than "searching" an account (see |
|
| Search accounts on a remote application. This operation is invoked:
A page or a list of accounts is expected to be returned. |
|
| Discover accounts on a remote application. This operation is invoked when an "account discovery task" is executed. |
|
Performing a Paged Search
A CSV file is not the only data source available when performing inbound import operations; it is also possible to import objects retrieved from a REST application. All the objects cannot be retrieved in a single call, because there may be hundreds of thousands of them. Objects must thus be retrieved page by page; a single REST application call will return a page of objects of a “reasonable” size, e.g. 500.
When performing a paged search with the REST Connector, a paging API is exposed in Groovy scripts, based on a page index incremented between consecutive page searches. The current page index can be retrieved as follows in a Groovy Rule: PAGE.index
However, instead of a simple index, some REST applications rely on an opaque token to keep state information between consecutive page searches. The PAGE API also exposes the PAGE.pagingToken
property, holding the token returned by the application in the previous page search. This token must then be sent to the application in the next page search request.
The next section lists the API enabling to search pages of objects: PageRequest
and PageResult
.
PageRequest API
/**
* Enable to keep track of the paged search state between consecutive search iterations.
*/
class PageRequest {
/**
* The page index, starting with 0, incremented by 1 at each search iteration.
*/
private final int index;
/**
* An offset indicating how many elements have been retrieved so far, it is an accumulation of the search iterations.
*/
private final int offset;
/**
* An alternative to {@link #index}: an opaque token returned by the REST application enabling the application to
* keep state information between consecutive paged searches. It must be sent back to the application when searching
* the next page.
*/
private final Object pagingToken;
}
PageResponse API
/**
* The result of a paged search.
*/
class PageResponse {
/**
* Return the page content, indicating that the search is complete.
*
* @param content a page of remote objects
* @return the page result
*/
public static PageResult complete(List<?> content);
/**
* Return the page content, indicating that the search is not complete; there are some entries left.
*
* @param content a page of remote objects
* @return the page result
*/
public static PageResult partial(List<?> content);
/**
* Return the page content, indicating that the search is not complete; there are some entries left.
*
* @param content a page of remote objects
* @param pagingToken an opaque token returned by the application to keep track of the search progress,
* to be sent in the next page request
* @return the page result
* @see PageRequest#getPagingToken()
*/
public static PageResult partial(List content, Object pagingToken);
}
}
A typical page search relying on a token is:
def pagingToken = PAGE.pagingToken as String
def response as Response
if (pagingToken) {
response = <send search request with paging token>
} else {
// Very first page request
response = <send search request without paging token>
}
def accounts = <extract entries from response>
def newPagingToken = <extract token from response>
if (newPagingToken) {
// The search is not over, there are some entries left
return PageResult.partial(accounts, newPagingToken)
} else {
return PageResult.complete(accounts)
}
Interrupting a Provisioning Operation
A RestConnectorException
may be thrown by Groovy scripts when the remote REST application returns an error or an unexpected response.
Throwing this exception has the following consequences:
the provisioning operation fails immediately
the exception's message is present in:
the account information UI
reporting data
audit logs
technical logs
Here is an example of the exception usage with a simple text error message:
REST.get(ACCOUNT_ID) {
onStatusCode 404, { throw new RestConnectorException('Cannot find account with id: ' + ACCOUNT_ID) }
}
Here is another example with an i18n error message:
REST.get(ACCOUNT_ID) {
onStatusCode 404, { throw new RestConnectorException('Cannot find account with id: ' + ACCOUNT_ID, "ui.errors.connectors.rest.accountNotFound.msg")
.putI18nArg('account_id', ACCOUNT_ID)
}
}
Response Parsing
TODO
TLS Trust Store
Memority provisions remote REST Applications via the https
protocol. This implies that the TLS certificate of remote REST Applications is trusted by Memority, so that the TLS connection can be established.
More specifically, this means that the Certificate Authority (CA) that emitted the REST Application’s TLS certificate must be imported into the Memority Trust Store.
There is a Trust Store per tenant, configurable via standard tenant Settings.
To configure trusted CA certificates, import the CA certificates as a PEM chain into the tenant Setting with key sync.ssl.trust.trustStore
.
The system properties toolkit.ssl.trust.enabled
and toolkit.ssl.trust.reload-enabled
must be set to true
so that the per-tenant Trust Store based on Settings is taken into account.
Sample Groovy Scripts
This section shows examples of Groovy scripts for the various connector operations.
GET_OBJECT Operation
// The Groovy binding "ACCOUNT_ID" contains the id of the Person to fetch
REST.get(ACCOUNT_ID) {
onStatusCode 404, {/* do something about the error */ }
}.json
SEARCH_OBJECTS Operation
// The Groovy binding "SEARCH_PARAMS" contains the search criteria provided by Synchronzation's (service) SimpleLookupStrategyDefault
// A single person is searched
def accountId = SEARCH_PARAMS.uid as String
LOG.debug('About to search single account with id {}', accountId)
REST.get(accountId).json
PAGED SEARCH_OBJECTS
This is a variant of a search operation where a “token-based” pagination is used, because in this use case the REST Connector is configured to perform an inbound import (not an outbound provisioning) for which a large population of accounts must be fetched. The whole population cannot be retrieved in a single request, the “search” operation is thus executed until no more results are left.
// In this example the pagination token expected by the remote application is a Map structured as {"index": index} where index is an int
// For another application it could be a simple string holding some kind of "state cookie"
def pagingToken = PAGE.pagingToken as Map<String, Integer>
def index = pagingToken?.index ?: 0
List accounts = REST.get('?index=' + index + '&size=5').json as List
if (accounts.size() == 5) {
// Generate the next paging token
return PageResult.partial(accounts, ["index": index+1])
} else {
// Search complete
return PageResult.complete(accounts)
}
DISCOVER_OBJECTS Operation
When an “account discovery” task is executed, the DISCOVER_OBJECTS
operation is executed. In the following example, since the population to discover is potentially large, a “classic” paginated search is performed. The search state is maintained
def index = PAGE.index ?: 0
def size = PAGE.size ?: 5
List accounts = REST.get('?index=' + index + '&size=' + size).json as List
if (accounts.size() == size) {
return PageResult.partial(accounts)
} else {
return PageResult.complete(accounts)
}
CREATE_OBJECT Operation
// The Person to create. The Groovy binding "OBJECT" contains the Person's attributes computed by Synchronzation service
def person = [
uid: OBJECT.uid,
givenName: OBJECT.givenName,
surname: OBJECT.surname,
birthDate: OBJECT.birthDate,
address:OBJECT.address
]
// The remote application expects the Person's attributes to be located under the root "data" key
REST.post('/') { json([data : person]) }
// Add the person to groups
OBJECT.groups?.each {
REST.post('/' + person.uid + '/~groups/' + it)
}
// By convention the Person's id must be returned upon the Person creation
OBJECT.uid
PATCH_OBJECT Operation
import com.memority.citadel.shared.api.im.operation.AttributeOperation
import com.memority.citadel.shared.api.im.operation.AttributePatch
import com.memority.domino.shared.api.AttributeDelta
import com.memority.toolkit.rest.client.groovy.api.Response
/*
OBJECT_ID -> CORRELATION_KEY (email)
OBJECT -> Values from application.xml mapping
PATCH -> Modified attributes only
*/
LOG.debug('Provisioning - UPDATE - CORRELATION_KEY {}', OBJECT_ID)
List<Map> attributes = new ArrayList()
def updateUser = [
"objectId" : OBJECT_ID,
"attributes": attributes
]
((Map<String, AttributeDelta>) PATCH).each {
String attribute, AttributeDelta attributeDelta ->
/*
* Complete the attrDelta.operation and check its value
* In if condition you will check if the operation is a DELETE
*/
// -> Should be like this
if (attrDelta.multiValuedState == AttributeDelta.MultiValuedState.MoNO){
attributes.add([
"operation": attributeDelta.operation == AttributeOperation.DELETE ? "REMOVE" : "SET",
"id" : attribute,
"values" : attributeDelta.operation == AttributeOperation.DELETE ? [] : [attributeDelta.value]
])
}else if (attrDelta.multiValuedState == AttributeDelta.MultiValuedState.MULTI) {
if (attrName == "group" && attrDelta.valuesToAdd()) {
(attrDelta.valuesToAdd() as List<String>).each { String value ->
def groupAdd = [
"roleId" : value,
"groupMode": ["group"]
]
def responsePatchGroup = REST.patch('api/v1/user/' + OBJECT_ID + '/role/') {
header("Authorization: Bearer", bearerValue)
header("Cookie", cookie)
json(groupAdd)
}
}
}
}
if (updateUser.attributes) {
//Send PATCH Request
def response = REST.patch('identity-internal-ws-update') { json(updateUser) } as Response
if (response.statusCode != 200) {
LOG.error('Provisioning - UPDATE - CORRELATION_KEY {} - Request Body {}', OBJECT_ID, toJson(updateUser))
LOG.error('Provisioning - UPDATE - CORRELATION_KEY {} - Response Code {}', OBJECT_ID, response.statusCode)
LOG.error('Provisioning - UPDATE - CORRELATION_KEY {} - Response Body {}', OBJECT_ID, toJson(response.json))
LOG.audit('Provisioning - UPDATE - CORRELATION_KEY {} - Request Body {}', OBJECT_ID, toJson(updateUser))
LOG.audit('Provisioning - UPDATE - CORRELATION_KEY {} - Response Code {}', OBJECT_ID, response.statusCode)
LOG.audit('Provisioning - UPDATE - CORRELATION_KEY {} - Response Body {}', OBJECT_ID, toJson(response.json))
throw new RestConnectorException("Provisioning - UPDATE - CORRELATION_KEY - ERROR")
} else {
LOG.debug('Provisioning - UPDATE - CORRELATION_KEY {} - Request Body {}', OBJECT_ID, toJson(updateUser))
LOG.debug('Provisioning - UPDATE - CORRELATION_KEY {} - Response Code {}', OBJECT_ID, response.statusCode)
LOG.debug('Provisioning - UPDATE - CORRELATION_KEY {} - Response Body {}', OBJECT_ID, toJson(response.json))
}
}
DELETE_OBJECT Operation
import com.memority.citadel.shared.api.im.ApiObject
// Remove the person from its groups. The Groovy binding "OBJECT_ID" contains the id of the Person to delete
OBJECT.groups?.each {
REST.delete('/' + OBJECT_ID + '/~groups/' + it)
}
// Only to test the PARSER feature: retrieve the person to delete
def person = PARSER.parse(REST.get(OBJECT_ID).json) as ApiObject
LOG.info("Retrieved person to delete: {}", person.value("lastName"))
// Delete the person
REST.delete(OBJECT_ID)