Webanalytics Docs

How to activate Webanalytic tools
Tags:
Categories:

  2 minute read  

Analysis of the use of the Search Index

To get an overview of how the Search Index is used (e.g. number of visits in a period etc), an analysis of the Nginx Access Logs can be activated. For this purpose Matomo is installed and a cron job is set up to analyze the access logs once a day. The overview of the analysis can then be viewed in the Matomo-UI via the browser:

http://192.168.98.115:8923/matomo (set your host instead of the Vagrant-VM)

Requirements

Already covered by the other roles in oersi: Installation of MariaDB and Nginx.

Configuration

  • matomo_install: set to true to install Matomo
  • matomo_superuser_name: Username of the Admin
  • matomo_superuser_password: Password of the Admin
  • matomo_superuser_mail: E-Mail of the Admin
  • matomo_dbname: Name of the database for Matomo
  • matomo_dbuser: User for access to the Matomo database
  • matomo_dbpassword: Password of the database user

Access-Log-Analysis for OERSI Frontend requests

  • A call to a detail-page is a single request and can be found as single */_doc/* entry in the matomo-page-view-analysis.
  • An initial call to the search-page consists of multiple (8) requests to */_msearch and therefore causes multiple entries in the matomo-page-view-analysis.
  • Using the search-field or the author-search-field will process multiple (no. depends on typing) requests to */_msearch, because the underlying data is requested while typing. => also causes multiple entries in the matomo-page-view-analysis
  • Use of a filter is just a single request to */_msearch therefore causes just a single entry in the matomo-page-view-analysis.

Elasticsearch request logging & analysis

The elasticsearch requests to the public (read-only) endpoint (https://oersi.org/resources/api/search/) can be logged, including, path, body etc of the request and number of results. Logged requests will be persisted per date in (oersi-internal) indices oersi_backend_elasticsearch_request_log-${DATE} (Date-Format YYYY-MM-DD) and cleaned up automatically after a period of time. All requests can be accessed via an alias oersi_backend_elasticsearch_request_log.

Additionally there is a script that runs once a day and analyzes the logged requests of the day before - it extracts, for example, things like search-terms from the requests (which may be provided at several different locations). Afterwards this is stored in the same index and record in the field requestAnalysis. This may be used for a visualization of the request-usages like search-terms (elasticsearch-aggs or even a Kibana visualization may be possible).

Configuration

  • This feature can be activated via feature toggle search_index_backend_features_log_elasticsearch_requests: "true".
  • Cleanup configuration via search_index_backend_log_elasticsearch_requests_cleanup_age

example: Top 100 search-terms

GET oersi_backend_elasticsearch_request_log/_search
{
  "size": 0, 
  "aggs": {
    "search": {
      "terms": {
        "field": "requestAnalysis.searchTerms.keyword",
        "size": 100
      }
    }
  }
}