Reloading solr core does not load mounted managed-schema

[KatiRG]

I followed the steps for binding using your own configuration files: https://github.com/ckan/ckan-solr?tab=readme-ov-file#use-your-own-configuration-files, but cannot get solr to see the custom managed-schema file.

After running the container with a bind mount (step 4), I inspected the container with docker inspect <container id> and confirmed that the mount had been made:

"Mounts": [
                {
                    "Type": "bind",
                    "Source": "/usr/lib/ckan/default/src/my_conf",
                    "Target": "/opt/solr/server/solr/configsets/ckan/conf"
                }
            ],

I then edited two files in /usr/lib/ckan/default/src/my_conf: the managed-schema file and the solrconfg.xml file (step 5). However, I noticed in the output of the inspection, that the managed-schema still points to the original container path and not to the mounted path:

"Env": [
                "PATH=/opt/solr/bin:/opt/solr/docker/scripts:/opt/solr/prometheus-exporter/bin:/opt/solr/cross-dc-manager/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "JAVA_HOME=/opt/java/openjdk",
                "LANG=en_US.UTF-8",
                "LANGUAGE=en_US:en",
                "LC_ALL=en_US.UTF-8",
                "JAVA_VERSION=jdk-17.0.14+7",
                "SOLR_USER=solr",
                "SOLR_UID=8983",
                "SOLR_GROUP=solr",
                "SOLR_GID=8983",
                "SOLR_INCLUDE=/etc/default/solr.in.sh",
                "SOLR_HOME=/var/solr/data",
                "SOLR_PID_DIR=/var/solr",
                "SOLR_LOGS_DIR=/var/solr/logs",
                "LOG4J_PROPS=/var/solr/log4j2.xml",
                "SOLR_JETTY_HOST=0.0.0.0",
                "SOLR_ZK_EMBEDDED_HOST=0.0.0.0",
                "SOLR_CONFIG_DIR=/opt/solr/server/solr/configsets",
                "SOLR_SCHEMA_FILE=/opt/solr/server/solr/configsets/ckan/conf/managed-schema"
            ],

I reloaded the core using the Solr admin page (step 6) anyway then tried to update a dataset, but this failed because the application is looking at the original managed-schema fields and not the customized ones in the source directory /usr/lib/ckan/default/src/my_conf.

Any advice?

[KatiRG]

I think the directory in the README steps is incorrect. Should be:

/opt/solr/server/solr/configsets/ckan/conf and not /opt/solr/server/solr/ckan/conf.

Can check by entering into the container (docker exec -u 0 -it ckan-solr bash) and navigating to the path.

[amercader]

Thanks @KatiRG this was fixed in 235381b. The paths might have changed between Solr 8 and 9

[amercader]

@KatiRG testing this further, I also noticed that Reloading the core via the UI didn't reflect changes made in the local schema. :(

I had to stop and re-run the container again for the changes to be picked up. I'm not sure if it's a permissions issue or just how Solr is expected to work. This needs a bit more investigation.

[benjwadams]

I've had to run a custom script to switch over to a ClassicIndexSchemaFactory to load a custom schema.xml with some custom fields and then had to switch back to managed schemas in order to support the dynamic field names when indexing new datasets: https://github.com/ioos/catalog-docker-base/blob/main/contrib/system_scripts/install_solr_schema.bash

[KatiRG]

What works for me (tested this several times now), is a hybrid of the two methods explained in the readme.

First, I edit the Dockerfile so that the ENV SOLR_SCHEMA_FILE points to our managed-schema url:

ENV SOLR_SCHEMA_FILE="https://github.com/ongov/ckanext-ontario_theme/blob/develop_prevalid_ckan2.11/config/solr/managed-schema"

and also make sure the ENV SOLR_CONFIG_DIR has the correct path: ENV SOLR_CONFIG_DIR="/opt/solr/server/solr/configsets".

Then

• make build
• make directory my_conf in /usr/lib/ckan/default/src
• copy the custom files we need in my_conf, such as managed-schema, protwords.txt, solrconfig.xml, stopwords.txt, and synonyms.txt. (make sure managed-schema.xml has the correct version <schema name="default-config" version="2.8">)
• bind the my_conf directory to the container:

docker run -p 8983:8983 --mount type=bind,source="$(pwd)",target=/opt/solr/server/solr/configsets/ckan/conf -d ckan/ckan-solr:2.11-solr9

• reload core in http://localhost:8983/solr/#/~cores/ckan and check that it is using the custom managed-schema: http://localhost:8983/solr/#/ckan/files?file=managed-schema

[niconoe]

I've struggled like crazy with this issue for my setup based on Docker-compose, here is a solution that appears to work:

• Create my schema in my-schema/managed-schema
• Use the following service configuration in Docker-compose:

solr:
    networks:
      - solrnet
    image: ckan/ckan-solr:${SOLR_IMAGE_VERSION}
    volumes:
      - solr_data:/var/solr
      - ./my-schema/managed-schema:/var/solr/data/ckan/conf/managed-schema:ro
    restart: always
    healthcheck:
      test: ["CMD", "wget", "-qO", "/dev/null", "http://localhost:8983/solr/"]

@KatiRG solution also worked for me, but required much more manual interventions.