DE EN EN (Google)

Export

This class defines an export job. Exports are user-bound (see /api/export).

An export defines:

The export is also used to track the job’s progress:

Finally, the export also provides information about the generated files and downloads:

Detailed information

Exports are stored on the server in their separate directories. Exports can contain folders and files, depending on the export definition. Exports are transported by the user using transport definitions.

After each sucessful Export an event is created (/api/event).

Lifecycle

The Export lifecycle looks as follows:

  1. Inform frontends using /api/event about the starting Export
  2. Log in the User of the Export, use the rights of that User
  3. Create a unique Export directory, in case of an update, wipe the existing one
  4. Execute the search of the Export
  5. Collect all Objects and the corresponding files (Assets)
  6. Copy all files into the Export filestructure
  7. Link files to the Object Information (relative local URLs to the exported files)
  8. Write XML and CSV files to the Export directory
  9. Start all defined transports to send emails and copy files
  10. Inform frontends using /api/event about the finished Export

Attributes

Name Description
_basetype Name of the base type (string, r): export
_date_created Timestamp when this export was created (timestamp, r)
_state State of the export (string, r): see state for a list of possible values
_schedules Configure when to launch the export (array of schedules, rw, optional, nullable)
  • if not set or set to null, the export is executed as soon as possible
  • _transports Transports defined for this export (array of transport export attributes, rw, optional)
    _log Export log (array of log events, r), possible types:
    • EXPORT_INSERT: export is created
    • EXPORT_START: export is started
    • EXPORT_FINISH: export is finished
    • EXPORT_FAILED: export aborted due to an error
    • EXPORT_STOPPED: export aborted by user
    _files List of files created in this export (array of files, r)
    export Export attributes:
    _id Export ID (integer, unique, r)
    _version Export version (integer, rw)
    type Type of export (string, rw, optional, defaults to export). Allowed values are export, export_incremental and download.
    name Export name (string, rw, optional): required if _schedules is set; defaults to "easydb-<type>-<_id>"
    search The search definition to find the objects (json, rw)
    fields Export configuration for specific fields. Map keys are the field names, map values are described in field export attributes, rw, optional, nullable)
    If set, only export the given fields. Otherwise, export all fields with standard configuration
    eas_fields Export configuration for EAS fields. Map keys are the field names, map values are described in EAS field export attributes(rw, optional, nullable))
    If unset, or set to null or empty map no files will be exported.
    classes Version attributes for each file class (map "<class>": Array of version export attributes, rw, optional)
    <class> is one of image, audio, video, office, or other
    assets Specific asset configuration (map: "<asset_id>": <value>, rw, optional) - <value> can be:
    • true: the corresponding Class Export Attributes are used to export this asset. This can be used to explicitly export a non-preferred version of an asset
    • false: this asset will not be included in the export. This can be used to not export preferred versions of an asset
    • asset export attribute. This can be used to explicitly export a non-preferred version of an asset
    csv CSV output configuration (rw, optional) - its value can be:
    merge_linked_objects which linked objects are exported in XML exports (string, rw, optional) - its value can be:
    • “none”: don’t include any linked objects (default value)
    • “not_in_main_search”: only load and export linked objects that are not in the main search (data model option)
    • “not_in_main_search_and_reverse”: like “not_in_main_search”, but also include reverse nested main objects
    merge_max_depth If linked objects are merged into XML, to which depth should links be resolved? (integer, rw, 1 - 9, default: 1)
    xml Include the data as XML (boolean, rw, optional): defaults to false
    xml_one_file_per_object Generate one XML file per object (boolean, rw, optional): defaults to false. If you want to apply a mapping, this option needs to be true.
    json Include data as JSON (boolean, rw, optional): defaults to false
    json_one_file_per_object Generate one JSON file per object (boolean, rw, optional): defaults to false
    xslt_xml_post_processing
    ↦↦ _id Asset ID to fetch XSLT stylesheet. Expected is an asset of mime type text/xsl which is used to post-process the generated XML files.
    mapping Mapping profile or profile id used to process the XML and CSV (string/int, rw, optional): defaults to the standard mapping “easydb”, “easydb” generates a hierarchical standard XML according to the objecttype definition (filtered by the mask), “easydb_flat” generates a flattened standard XML according to the objecttype definition (filtered by the mask). If a mapping profile is applied, the id of the profile is returned.
    all_languages Export all multilanguage fields in all existing database languages. This option is used for XML and CSV export.
    • false: use data languages configured by user (default)
    • true: use all data languages activated in base configuration
    batch_size Export-specific batch size (integer, rw, optional). If unset, system-wide configuration is used. 0 indicates unlimited batch size, positive integers may be used to set the batch size.
    filename_original Use original filename (boolean, rw, optional). If set to true, the original filename base is used for exported assets.
    filename_template Filename template (string, rw, optional). If a string is supplied, it is used as filename base template for the exported assets. The string may contain variables as in per-objecttype filenames. (filename_original=true and filename_template="..." can’t be set at the same time)
    produce_options Options to be passed to export produce plugins (object, rw, optional).
    flat If set, no directory structure is used for exported files (boolean, rw, optional, defaults to false)

    * Fields are provided in the ES format (the one used by /api/search)

    Field Export Attributes

    The field export attributes is a JSON object with the following optional arguments.

    Parameter Value Description
    display_name String If set, replace the name of the field by this value in the export CSV and XML.

    EAS Field Export Attributes

    The field export attributes is a JSON object with the following optional arguments. Some of them are only meaningful for certain EAS fields:

    Parameter Value Description
    files Boolean If set, copy or produce export files for the assets found in the search result.
    data Array of Maps One element for each version.
    type "original" Export Link to the original.
    "current" Link to the current original.
    "version" Link to a given version.
    format "long" A more verbose format is used for export the URL and accompanying data.
    "short" A less verbose format is used for export the URL and accompanying data.

    type “version”

    Parameter Value Description
    version "<class>.<version_name>" Selects the given <version_name> in class <class> to export.

    Asset Export Attributes

    Parameter Value Description
    versions Array of version export attributes Version export information for the asset.
    eas_parent_id Integer For temporarily produced version, the link to the parent EAS id.
    json Boolean If set, per-export json option is overridden for this asset. If there are multiple assets per object, JSON output is enabled if at least one asset has json set to true.

    Version Export Attributes

    The version export attributes is a JSON object with the following optional arguments. Some of them are only meaningful for certain asset classes:

    Parameter Value Classes Description
    type "original" All Selects the original.
    "current" All Selects the current original, that is the rotateted version.
    "version" All Selects a version of the current original.
    "custom" All Produces a custom version of the current original.
    metadata "standard" image,audio,video The selection for the metadata mapping is used from the Pool or Objecttype setting. If metadata is unset, "standard" is assumed.
    "keep" image,audio,video The metadata is kept as it is. It is possible that elsewhere versions already got a Metatda-Profile written.
    "remove" image,audio,video The metadata is removed.
    <profile-id> image,audio,video The metadata is mapped using the Profile with id <profile-id>.

    type “version”

    Parameter Value Classes Description
    version "<version_name>" All Selects the given <version_name> to export. Available version names depend on the class.

    type “custom”

    Parameter Value Classes Description
    custom
    watermark Boolean image true produces a custom version including a watermark, this works only if version is unset, false does not include a watermark.
    format "keep" image The format of the image is unchanged to its current version format.
    "jpeg" image The format of the produced image will be JPEG.
    "tiff" image The format of the produced image will be TIFF.
    "png" image The format of the produced image will be PNG.
    "bmp" image The format of the produced image will be BMP.
    jpeg_quality "maximum"
    "medium"
    "small"
    image The quality of the JPEG, if format is JPEG.
    colorspace "keep"
    "rgb"
    "cmyk"
    "gray"
    image The colorspace of the image.
    size "keep" image The size of the produced version is the same as the current version (unchanged).
    "custom" image The size of the produced version is given by the custom_size parameters.
    custom_size "dimension_max" image The size of the long edge is set by custom_size_pixel.
    "dimension_min" image The size of the short edge is set by custom_size_pixel.
    "width" image The width is set by custom_size_pixel.
    "height" image The height is set by custom_size_pixel.
    custom_size_pixel Integer image The size of the custom_size value in pixels.
    transform Array of transformation Info image See in /eas/produce for more information. This parameter is allowed only when used in assets.

    Field Export Attributes

    Array of version export attributes: version attributes for the asset.

    CSV Export Attributes

    The CSV Export always includes a header with the field names. Values can include Newlines.

    Parameter Value Description
    delimiter Char Use this character as the delimiter between columns.
    quote Char Use this character to quote values. Values are always quoted, even if they wouldn’t have to be quoted.
    escape Char Use this character to escape the quote character within values.
    use_bom Boolean Use byte-order mark in file. false if not set.
    empty_columns Boolean Allow empty columns in exported CSV. false if not set.
    hierarchy String Export options for hierarchical objects. One of "column", "/", ">" or null. null if not set.

    Export of hierarchical linked objects

    If hierarchical objects are exported, there are multiple options to export the path in the CSV file:

    This rule applies for top level objects as well as for linked objects.

    Transport Export Attributes

    A Transport is used to send exports after they are finished.

    Parameter Description
    type Transport type (string, rw): currently email, rsync and download are supported
    packer Packer to be used (string, rw, optional): defaults to null
    • “zip”: ZIP archive, unlimited size
    • “zip_10mb”: ZIP archive(s), 10 MB limit per file (see below)
    • “zip_2gb”: ZIP archive(s), 2 GB limit per file (see below)
    • “tar_bz2”: TAR archive using BZip2 compression
    size limits are not very reliable and should be taken as a hint only:
    • splitting is done by input file size, compression or archiving overhead is ignored, archives can be smaller or even larger than input files
    • single files greater than size limit are not splitted, resulting archive file is larger than given size limit
    email Configuration for sending e-mails (object, rw):
    recipients Array of recipients, which can be user(short), group(short) or an ad-hoc user based on an e-mail
    message Optional message to attach to the standard e-mail text
    _state State of the transport (string, r): see state for a list of possible values
    _download Available download (download, r)
    options Transport attributes (object, rw, optional): depending on the type, additional attributes can be configured; see below

    Ad-hoc users are provided as in /api/collection.

    Transport type “download”

    This type of transport has no attributes, a list of downloads will be available in the read-only attribute _download.

    Transport type “rsync”

    RSYNC can be used to copy files away from the Export directory to a backup directory or a remote server using the SSH, RSH, or RSYNC as transport protocol.

    Parameter Description
    destination The rsync destination (string)
    options Additional rsync options (object, optional)

    File

    For each file, the following attributes are provided:

    Value Description
    path Relative path of the file (string)
    size File size, in bytes (integer)
    eas_id Asset ID of file (integer, optional)
    eas_version Asset version of file (string, optional)
    eas_fileclass Asset file class of file (string, optional)

    Download

    For each download, the following attributes are provided:

    Parameter Description
    _files List of files, zip32 has a 2GB filesize limit which might yield to multiple files for the donwload
    url Relative URL of the file, starting with a Secret-Passkey for direct access to the file (string)
    size If available, the size of the download (integer, optional)

    The url is persistent for the lifetime of the export’s transport. So that after a scheduled update of the export, the URL does not change.

    State

    These values are always possible for _state:

    Value Description
    new Created, not processed
    pending Export has been started but is not processed, yet
    processing In progress
    failed Failed
    done Completed without errors

    Example

    {
    	"classes": {
    		"image": {
    			"version": "current+original"
    		},
    		"office": {
    			"version": "current"
    		}
    	},
    	"eas_fields": [
    		"bilder.datei",
    		"bilder.rueckseite",
    		"bilder._nested:bilder__documents.file"
    	],
    	"assets": {
    		"1000127337": true,
    		"1000130848": {
    			"version": "custom",
    			"format": "tiff",
    			"size": "custom",
    			"custom_size": "height",
    			"custom_size_pixel": "123",
    			"watermark": true
    		},
    		"1000130849": false,
    		"1000130851": true
    	},
    	"csv": {
    		"delimiter": ";",
    		"quote": "\"",
    		"escape": "\"",
    		"empty_columns": false,
    		"all_languages": false,
    		"hierarchy": ">"
    	},
    	"merge_linked_objects": "not_in_main_search",
    	"merge_max_depth": "3",
    	"xml": true,
    	"xml_one_file_per_object": true,
    	"search": {
    		"search": [
    			{
    				"bool": "must",
    				"type": "in",
    				"fields": [
    					"bar._pool.pool._id",
    					"Test._pool.pool._id",
    					"video._pool.pool._id",
    					"bilder._pool.pool._id"
    				],
    				"in": [
    					1,
    					2,
    					35,
    					37,
    					38
    				]
    			}
    		],
    		"sort": []
    	},
    	"fields": {
    		"video.videodatei": {
    			"watermark": true,
    			"display_name": "videodatei",
    			"versions": [
    				"huge"
    			]
    		},
    		"bilder.standort.Standort._objecttype": {
    			"display_name": "_objecttype"
    		},
    		"bilder.standort.Standort._id": {
    			"display_name": "_id"
    		},
    		"bilder.standort.Standort.version": {
    			"display_name": "version"
    		},
    		"bilder.standort.Standort._changelog": {
    			"display_name": "_changelog"
    		},
    		"bilder.standort.Standort._standard": {
    			"display_name": "_standard"
    		},
    		"bilder.standort.Standort.ort": {
    			"display_name": "ort"
    		},
    		"bilder.standort.Standort.synonyme": {
    			"display_name": "synonyme"
    		},
    		"bilder.standort": {
    			"display_name": "standort"
    		}
    	}
    }
    

    Note: state has been implemented as _state directly in export and transport.