Import Task

Definition

An Import Task enables an end user, or the system when the import is scheduled, to upload a CSV file to import.

The import can begin immediately, or can be scheduled to a fixed date for a later execution (just once).

Usage

An Import task can be :

Private, meaning the task has been executed on demand by a end user
Public, meaning the task is executed periodically by a scheduled job

Launch an import - Private Import

When a task execution is "private", only the user who launched it can access the corresponding import task (other users with administrative rights may also access it).

	Steps	Preview
1	In the My Imports Section, click on the Run Task button
2	Select the Import to Run and upload the CSV file The format of the file must match the settings already configured in configuration (separator char, escaping char, etc.) The user cannot override those settings. File Format information are available to allow the user to know the expected format.
3	Optional : Click on the Simulate Button
4	Click on the Run Button If the user wants to plan the import execution for later he can click on the Plan button and select a date/time at which the import task will be executed. If not specified, the import starts immediately. The task execution is not recurrent; if scheduled, it is executed only once.

Visualize an import task - Public Import

When a task execution is "public", only the users with appropriate rights can retrieve the export files produced by the scheduled job execution.

	Steps	Preview
1	In the Public Imports Section, for a dedicated task, click on the Details button

Access Control

Execution rights can be granted on a per-task basis. For example, if two Import Tasks "A"and "B" are configured, a user may be allowed to execute "A" only, but not "B".

Administrative rights can also be granted so that a user can execute any Import Task (more on that later).

Access control is applied when executing an import task.

All these use cases are controlled using Permissions that follow the standard schemes:

Permissions

Permissions

Scope

Description

ImportTaskAccess:private

TASK

(per Synchronization Task Definition)

This permission is based on the SynchronizationTaskDefinition resource. This means that a User can have this permission to access task executions for some SynchronizationTaskDefinitions only, but not others.

Here the permission gives access to private task executions: can run, read, cancel executions that are launched by the user, either immediately or scheduled once.

Sync:run Sync:monitor

GLOBAL

Execution of any SynchronizationTaskDefinition.Retrieval of any SynchronizationTaskDefinition execution report.

These are GLOBAL permissions, i.e. they are not applied on a per-task basis, they apply to all tasks, similar to other configuration permissions in Citadel.

Those permissions are arranged in the following rights that can be granted to users:

Rights

Right

Permissions

Security Scope

Description

sys.sync-import-task-private

ImportTaskAccess:private

TASK

(per Synchronization Task Definition)

Give capacity to run a Synchronization Task Definition and create a new private Task execution.

sys.sync-task-run

Sync:run

GLOBAL

Run all Synchronization Task Definitions.

sys.sync-task-admin

Sync:run

Sync:monitor

GLOBAL

Run or retrieve all executions of Synchronization Task Definitions.

The "private" task execution permission enabling end users to retrieve an import report is also specific to the "import" feature (see next section). Such a "private" permission does not exist with a regular Synchronization Task.

Configuration

The execution of an Import Task is not to be confused with its configuration. Tasks are configured by administrators, and can then be executed by end-users (or by the system when scheduled import), if they have the appropriate execution right on the Import Task.

An Import Task is configured through a classic Synchronization Task. When executing an Import Task, a regular Synchronization Task is indeed launched behind the scene. The "import" flavor only stems from the fact that a CSV file is uploaded on-the-fly by an end user when launching or scheduling (just once) the Synchronization Task.

The main object is therefore a Synchronization Task, but in order to function, this Synchronization Task must reference the following elements:

An application, the main configuration entry point of a synchronization, which defines namely the mapping between IDM service attributes and remote attributes
- which itself references a connector, the component enabling Synchronization Task to communicate with the remote application
And possibly, but not necessarily, a Post Synchronization Task, to configure what will be done in term of post processing report.

Example

Synchronization Task Configuration Example

XML

<dmn:SynchronizationTaskDefinition id="importTask-partner" xmlns:dmn="http://www.memority.com/domino_sync/1_0">
      <name>Import Partner</name>
<description></description>
  <active>true</active>
  <allowCyclicInputData>false</allowCyclicInputData>
  <allowDuplicateInputData>false</allowDuplicateInputData>
  <applicationId>import-partner-application</applicationId>
  <bypassIdmDeduplication>false</bypassIdmDeduplication>
  <bypassIdmReferenceIntegrity>false</bypassIdmReferenceIntegrity>
  <bypassIdmUnicity>false</bypassIdmUnicity>
  <bypassIdmValidation>false</bypassIdmValidation>
  <objectClassId>__ACCOUNT__</objectClassId>
  <recoverable>false</recoverable>
  <shadowKind>ACCOUNT</shadowKind>
  <simulate>false</simulate>
  <skipCorrelationOnImport>false</skipCorrelationOnImport>
  <sortInputData>false</sortInputData>
  <taskType>IMPORT</taskType>
</dmn:SynchronizationTaskDefinition>

REST API

The Memority Synchronization service exposes the following Import REST API:

Full import REST API

Resource	Method	Body	Permission	Description	Response
All APIs under `{tenant}/api/sync/~run-sync-task` relate to instances management and require a `Sync:run` permission
`{tenant}/api/sync/{task-definition-id}/instances/~content`	POST multipart	JSON + FILE	Sync:run	Launch an import task by providing the identifier of an `SyncTaskDefinition` task configuration (path variable), an `SyncTaskRunOptions` object (JSON payload with key options) and a data file to import (FILE payload with key file) The execution of this method is asynchronous, it immediately returns to the caller an UUID, the task instance id, that can then be used in the API to get an execution report.	Success HTTP code 200. Body: a task instance id as a simple string (not JSON): an UUID identifier enabling to monitor the task progress, or cancel the running task (see below). Errors HTTP code 404 if the task configuration id is not found.
`{tenant}/api/sync/conf/sync-task-definitions/{task-definition-id}/~data-format`	GET	N/A	Sync:run	Return information about the expected CSV file format.	Success HTTP code 200. Body: Information about the expected CSV file format: header names, mandatory fields, fields format.
All APIs under `{tenant}/api/sync/~private/~run-sync-task` relate to private instances management and require an `ImportTaskAccess:private:?` permission
`{tenant}/api/sync/~private/{task-definition-id}/instances/~content`	POST	JSON + FILE	`ImportTaskAccess:private:{task-definition-id}`	Launch an import task by providing the identifier of an `SyncTaskDefinition` task configuration (path variable), an `SyncTaskRunOptions` object (JSON payload with key options) and a data file to import (FILE payload with key file) The execution of this method is asynchronous, it immediately returns to the caller an UUID, the task instance id, that can then be used in the API methods below.	Success HTTP code 200. Body: a task instance id as a simple string (not JSON): an UUID identifier enabling to monitor the task progress, or cancel the running task (see below). Errors HTTP code 404 if the task configuration id is not found.
`{tenant}/api/sync/~private/{task-definition-id}/instances/{task-instance-id}/~cancel`	POST	N/A	`ImportTaskAccess:private:{task-definition-id}`	Cancel a running task.	Success HTTP code 200 Body: empty. Errors HTTP code 404 if the task instance id is not found. HTTP code 409 ("conflict") if the task cannot be canceled because it is in a terminal state.
`{tenant}/api/sync/~private/{task-definition-id}/instances/{task-instance-id}`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Get the execution report of a task.	Success HTTP code 200 Body: JSON representation of the task execution report. Errors HTTP code 404 if the task instance id is not found.
`{tenant}/api/sync/~private/instances/`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Get all the visible execution reports. Reports can be filtered by passing the following optional HTTP parameters in the GET request: `filter` : string, part of the name (case insensitive) of a Task Definition whose reports are retrieved `kind` : choice among IDENTITY, ORGANIZATION OR RESOURCE, reports for this kind of objects are retrieved `expandErrorDetails` : boolean, if true all errors encountered during the import task execution are returned, not only a subset of the 20 first errors
`{tenant}/api/sync/conf/sync-task-definitions/~private/{task-definition-id}/~data-format`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Return information about the expected CSV file format.	Success HTTP code 200. Body: Information about the expected CSV file format: header names, mandatory fields, fields format.

Options can be provided in the JSON body when launching an import task, they are listed below:

Property Name	Type	Mandatory	Description	Values (default value in bold)
executionPlan	`Complex`	NO	Only set when the Task execution is deferred to a later date. This is the CRON expression defining when the task will be executed (only once). It can be executed only once, thus providing a recurrent CRON expression is forbidden.	-
simulate	`boolean`	NO	Whether the task execution is simulated or not.	false, true

Here is an example of a multi-part curl command used to launch an Import Task with a CSV file and the "simulate" option set to "true":

Launch an import task

CODE

curl -i -X POST \
-H "Content-Type: multipart/form-data" \
-u login:password \
-F "options={\"simulate\":\"true\"};type=application/json" \
-F "file=@mydata.csv" \
http://localhost:8150/memority/api/sync/~run-sync-task/myimportTask/~content

Here is an example of JSON returned by ~data-format:

Data format

CODE

{
  "metadata" : {
    "type" : "CSV",
    "encoding" : "utf-8",
    "uniqueAttributeId" : "uid",
    "quote" : "\"",
    "escape" : "\\",
    "commentMarker" : "#",
    "fieldDelimiter" : ",",
    "recordSeparator" : "\r\n"
  },
  "attributes" : [ {
    "attribute" : "age",
    "mandatory" : false,
    "multiValued" : false
  }, {
    "attribute" : "email",
    "mandatory" : false,
    "multiValued" : false
  }, {
    "attribute" : "login",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "nom",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "prenom",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "societe",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "titre",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "uid",
    "mandatory" : true,
    "multiValued" : false
  } ]
}

Usage

Launch an import - Private Import

Visualize an import task - Public Import

Access Control

Configuration

Example

REST API

Read Next