Solr Audit Logging

4 min readJun 19, 2021

From version 8.1 and above, Solr has the ability to log an audit trail of all HTTP requests entering the system. Audit loggers are pluggable to suit any possible format or log destination.

This article details how to enable audit logging and covers a few auditing test cases. Solr version used for testing this feature is : 8.4.1

Why this blog?

There are official documentation from solr to enable to audit logging here but it lacks the integration of Authentication, Authorization and Audit logging in a single configuration file which is the actual need for auditing at the first place for most of the users.

In this blog we have tried to cover the configuration and few important use cases of auditing to get you started.

Configuration change (Enable Authentication, Authorization and Audit logging)

To Enable Basic Authentication , Role Based Authorization and Audit logging , create/update security.json and upload to zookeeper:

{
  "authentication": {
    "class": "solr.BasicAuthPlugin",
    "blockUnknown": true,
    "credentials": {
      "admin-user": "IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
      "dev-user": "IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="
    }
  },
  "authorization": {
    "class": "solr.RuleBasedAuthorizationPlugin",
    "user-role": {
      "admin-user": "admin",
      "dev-user": "dev"
    },
    "permissions": [
      {
        "name": "dev-private-collection",
        "collection": "dev-private",
        "role": "dev"
      },
      {
        "name": "security-read",
        "role": "admin"
      },
      {
        "name": "security-edit",
        "role": "admin"
      }
    ]
  },
  "auditlogging": {
    "class": "solr.SolrLogAuditLoggerPlugin"
  }
}

Audit Logging Test Cases:

Note: For the below test cases we need to make sure the logs has an entry for the user who is performing these operations. So the user used is highlighted on the Solr UI and the same user can be seen in the logs as well right below it.

Case 1: Select Query auditing

2020-01-22 08:51:35.177 INFO  (qtp1850180796-93) [c:techproducts s:shard1 r:core_node5 x:techproducts_shard1_replica_n2] o.a.s.c.S.Request[techproducts_shard1_replica_n2]  webapp=/solr path=/select params={q=*:*&_=1579681921602} hits=55 status=0 QTime=532020-01-22 08:51:35.177 INFO  (audit-30-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" 
message="Completed" method="GET"status="200" requestType="SEARCH" username="admin-user" resource="/select" queryString="_=1579681921602&q=*:*" collections=[techproducts]

2020-01-22 08:53:34.370 INFO  (qtp1850180796-264) [c:techproducts s:shard1 r:core_node5 x:techproducts_shard1_replica_n2] o.a.s.c.S.Request[techproducts_shard1_replica_n2]  webapp=/solr path=/select params={q=id:"0812521390"&_=1579681921602} hits=1 status=0 QTime=362020-01-22 08:53:34.370 INFO  (audit-30-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="GET"status="200" requestType="SEARCH" username="dev-user" resource="/select" queryString="_=1579681921602&q=id:%220812521390%22"collections=[techproducts]

Case 2: Update Query auditing

2020-01-22 09:04:44.233 INFO  (audit-30-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="POST"status="200"requestType="UPDATE" username="dev-user" resource="/update"queryString="update.distrib=FROMLEADER&distrib.from=http%3A%2F%2F172.30.126.9%3A8983%2Fsolr%2Ftechproducts_shard1_replica_n1%2F&wt=javabin&version=2" collections=[techproducts]2020-01-22 09:04:44.237 INFO  (qtp1850180796-337) [c:techproducts s:shard2 r:core_node8 x:techproducts_shard2_replica_n6]o.a.s.u.p.LogUpdateProcessorFactory [techproducts_shard2_replica_n6]  webapp=/solr path=/updateparams={commitWithin=1000&overwrite=true&wt=json&_=1579683642208}{add=[123456]} 0 382020-01-22 09:04:44.237 INFO  (audit-30-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="POST"status="200" requestType="UPDATE" username="dev-user" resource="/update"queryString="_=1579683642208&commitWithin=1000&overwrite=true&wt=json" collections=[techproducts]

Case 3: Create Collection auditing

2020-01-22 09:16:43.992 INFO  (audit-30-thread-1) [   ]o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="GET" status="200"requestType="ADMIN" username="admin-user" resource="/admin/collections" queryString="_=1579681887628&action=CREATE&autoAddReplicas=false&collection.configName=gettingstarted&maxShardsPerNode=1&name=test_collection&numShards=1&replicationFactor=2&router.name=compositeId&wt=json" collections=[test_collection]

Case 4: Delete Collection auditing

Case 5: Reload Collection auditing

2020-01-22 09:26:51.087 INFO  (audit-30-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="GET" status="200"requestType="ADMIN" username="admin-user" resource="/admin/collections" queryString="_=1579681887628&action=RELOAD&name=techproducts&wt=json"collections=[techproducts]

Case 5: Config API auditing

Retrieving the Config via config api

2020-01-22 10:08:39.965 INFO  (qtp1850180796-10755) [c:techproducts s:shard1 r:core_node3 x:techproducts_shard1_replica_n1]o.a.s.c.S.Request [techproducts_shard1_replica_n1]  webapp=/solr path=/config params={} status=0 QTime=02020-01-22 10:08:39.966 INFO  (audit-74-thread-1) [   ] o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED"message="Completed" method="GET" status="200" requestType="UNKNOWN" username="dev-user" resource="/config" queryString="null"collections=[techproducts]

Case 6: Modify Config auditing

Modify config commands to modify

m-abc:solr-8.4.1 dinesh$ curl --user <user>:<pass> -X POST -H 'Content-type: application/json' -d'{"set-property":{"updateHandler.autoCommit.maxTime":10000}}' http://localhost:8983/solr/techproducts/config{"responseHeader":{"status":0,"QTime":925},"WARNING":"This response format is experimental.  It is likely to change in the future."}2020-01-22 12:05:31.784 INFO  (audit-74-thread-1) [   ]o.a.s.s.SolrLogAuditLoggerPlugin type="COMPLETED" message="Completed" method="GET" status="200" requestType="UNKNOWN"username="dev-user" resource="/config/znodeVersion" queryString="overlay=5&wt=javabin&version=2" collections=[techproducts]

Metrics

AuditLoggerPlugins record metrics about count and timing of log requests, as well as queue size for async loggers.

Conclusion

Auditing is one of the important requirement for Health Insurance Portability and Accountability Act of 1996 (HIPAA) and the Sarbanes-Oxley Act of 2002 (SOX) compliance and details mentioned in this blog can help simplify the steps needed to enable auditing in your solr cluster.