Monitoring Docs

Monitoring Import workflows

You can activate a monitoring of the etl / import-script processes and receive a daily report that includes number of sucesses and failures for each process.

Requirements

mail (mailutils) needs to be available and configured on the system.

Configuration

See ansible/group_vars/all.yml

• activate monitoring via search_index_monitoring: true
• set sender address via search_index_monitoring_from_address: your-sender@somewhere
• set recipient addess via search_index_monitoring_recipients: your-recipient1@somewhere,your-recipient2@somewhere
• (optional) set failure-bound for the report via search_index_monitoring_import_failure_bound: <number> (send ETL report when number of failures is greater than or equal to this bound, default: 0)

Metadata statistics

To avoid unnecessary amounts of data, small statistics of the existing metadata in the index can be activated. Thus not all old data must be kept up to pursue the development of the metadata. The statistic is stored in elasticsearch (index oer_data_statistics) and determines per source (and total) how many data are available and if certain fields are set.

It is only checked if the field is set, not how many values are present in the field.

If an OER is present in several sources, it can come to interactions in the statistics (in the data it is not recognizable from which source a field comes).

Configuration

See ansible/roles/index-scripts/defaults/main.yml

• activate statistics via search_index_statistics: true
• set fields to include in search_index_statistics_fields
• schedule process via search_index_statistics_schedule_* variables