Get 50% discount from my Spring Data book! Use campaign code BIGW50. The deal lasts only 4 days so be fast!

Running Solr with Maven

Share
Search Bot

Solr is an open source search server which is built by using the indexing and search capabilities of Lucene Core, and it can be used for implementing scalable search engines with almost any programming language.

Even though Solr has many advantages, setting up a a development environment is not one of them. This blog entry describes how we can run Solr by using Maven and ensure that each developer uses the same configuration, schema and Solr version.

The requirements of our Maven build are following:

  • The properties of our Maven build must be read from an external property file. The only exception to this rule is that the version numbers of the dependencies are declared in our POM file.
  • The build process must copy the Solr configuration files to the correct directory when our Solr instance is started.
  • The build process must clean up the configuration files when a developer executes mvn clean command at command prompt.
  • It must be possible to start our Solr instance by using the Jetty Maven plugin.

We can fulfil these requirements by following these steps:

  1. Create a POM file.
  2. Get the required dependencies.
  3. Get the Solr configuration files.
  4. Create the properties file which contain the properties used in our Maven build.
  5. Edit the solr.xml file.
  6. Configure the Properties Maven plugin.
  7. Configure the Copy Maven plugin.
  8. Configure the Jetty Maven plugin.

These steps are described with more details in the following.

Creating the POM file

First, We have to create a POM file for a web application project. The skeleton of our POM file looks as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>net.petrikainulainen.maven</groupId>
    <artifactId>running-solr-with-maven</artifactId>
    <packaging>war</packaging>
    <version>0.1</version>
   
    <profiles>
        <!-- Add profile configuration here -->
    </profiles>
    <dependencies>
        <!-- Add dependencies here -->
    </dependencies>
    <build>
        <finalName>solr</finalName>
        <!-- Add filter configuration here -->
        <!-- Add resources configuration here -->
        <plugins>
            <!-- Add plugin configuration here -->
        </plugins>
    </build>
</project>

Getting the Required Dependencies

We need to configure the following dependencies in our pom.xml file:

  • SLF4J
  • SLF4J interceptors for both java.util.logging (JUL) and java.commons.logging (JCL) logging frameworks.
  • SLF4J Log4j 1.2.x binding
  • Log4j
  • Solr 4.3.0 (war)

Note: The reason why we have to configure the logging jars as dependencies is that the logging setup of Solr was changed when Solr 4.3.0 was released.

In other words, we have to add the following dependency declarations to the dependencies section of our POM file:

<!-- SLF4J -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.5</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jcl-over-slf4j</artifactId>
    <version>1.7.5</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
    <version>1.7.5</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.7.5</version>
</dependency>
<!-- Log4j -->
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>
<!-- Solr 4.3.0 -->
<dependency>
    <groupId>org.apache.solr</groupId>
    <artifactId>solr</artifactId>
    <version>4.3.0</version>
    <type>war</type>
</dependency>

Getting the Solr Configuration Files

We can get the Solr configuration files by following these steps:

  1. Download the binary distribution of Solr 4.3.0.
  2. Extract the downloaded package to the desired directory.
  3. Go the root directory of the extracted Solr binary distribution.
  4. Copy the following files from the directory example/solr/collection1/conf to the directory src/main/config: admin-extra.html, admin-extra-menu.menu-bottom.html, admin-extra.menu-top.hml, currency.xml, elevate.xml, mapping-FoldToASCII.txt, mapping-ISOLatin1Accent.txt, protwords.xml, schema.xml, scripts.conf, solrconfig.xml, spellings.txt, stopwords.txt, synonyms.txt and update-script.js.
  5. Copy the language specific configuration files found from directory example/solr/collection1/conf/lang to the directry src/main/config/lang.
  6. Copy the Velocity macros and other files found from the directory example/solr/collection1/conf/velocity to the directry src/main/config/velocity.
  7. Copy the XSL style sheets found from the directory example/solr/collection1/conf/xslt to the directry src/main/config/xslt.
  8. Copy the solr.xml file from the directory exaple/solr/collection1 to the directory src/main/resources.
  9. Create a directory src/main/webapp/WEB-INF. This directory is required so that the Solr instance can be started.

We have now successfully obtained the required files and are ready to move on to the next phase.

Creating the Properties File

Our next is the to create the properties file that is used in our Maven build and add the required build profile configuration to our POM file. Lets move on and find out how this is done.

First, we have create the properties file which is used in our Maven build. We can do this by following these steps:

  1. Create directory profiles/dev to the root directory of our Maven project.
  2. Create a properties file config.properties to the profiles/dev directory.

Our properties file has three properties which are described in the following:

  • The solr.detault.core.directory property states the value of the default core directory. This is a directory which is created under the home directory of our Solr instance. This directory stores the configuration of our Solr instance and its data.
  • The solr.default.core.name property states the name of the default core.
  • The solr.solr.home property states the home directory of our Solr installation.

The content of the config.properties file looks as follows:

#SOLR PROPERTIES
#Configures the directory used to store the data and configuration of the Solr default core
solr.default.core.directory=todo
#Configures the name of the Solr default core.
solr.default.core.name=todo

#SYSTEM PROPERTIES
#Configures the home directory of Solr. Set the preferred directory path here.
solr.solr.home=

Second, we must configure the build profiles of our Maven build and use filtering to replace replace the variables included in our resources. We can do this by following these steps:

  1. Create a single profile called dev and ensure that it is the default profile of our build.
  2. Declare a property called build.profile.id and set its value to ‘dev’.
  3. Create a filter that reads the profile specific configuration file and replaces the variables found from our resources with the actual property values.

We can finish steps one and two by adding the following profile declaration to our POM file:

<profile>
    <id>dev</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
        <build.profile.id>dev</build.profile.id>
    </properties>
</profile>

We can finish step three by adding the following XML to the build section of our POM file:

<filters>
    <filter>${project.basedir}/profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
    <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Editing the solr.xml File

Because we use a profile specific configuration file to configure the name and the instance directory of the Solr default core, we have to make changes to the solr.xml file. These changes are described in the following:

  1. The value of the solr.default.core.name property must be set as the value of the defaultCoreNameAttribute attribute of the cores element.
  2. The value of the solr.default.core.name property must be set as the value of the name attribute of the core element.
  3. The value of the solr.default.core.directory property must be set as the value of the instanceDir attribute of the core element.

The content of the solr.xml file looks as follows:

<solr persistent="true">
  <cores adminPath="/admin/cores" defaultCoreName="${solr.default.core.name}" host="${host:}" hostPort="${jetty.port:}" hostContext="${hostContext:}" zkClientTimeout="${zkClientTimeout:15000}">
    <core name="${solr.default.core.name}" instanceDir="${solr.default.core.directory}" />
  </cores>
</solr>

Configuring the Properties Maven Plugin

Because we want that all property values used in our POM file are read from an external properties file, we have to use a plugin called the Properties Maven plugin. We can configure this plugin by following these steps:

  1. Ensure that the properties are read from the profile specific configuration file.
  2. Create an execution that runs the read-project-properties goal of the Properties Maven plugin in the initialize phase of the Maven default lifecycle.
  3. Create an execution that runs the read-project properties goal of the Properties Maven plugin in the pre-clean phase of the Maven clean lifecycle.

The configuration of the Properties Maven plugin looks as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>properties-maven-plugin</artifactId>
    <version>1.0-alpha-2</version>
    <configuration>
        <files>
            <!-- Properties are read from profile specific property file -->
            <file>${project.basedir}/profiles/${build.profile.id}/config.properties</file>
        </files>
    </configuration>
    <executions>
        <!-- Load properties for the default lifecycle -->
        <execution>
            <id>default-lifecycle-properties</id>
            <phase>initialize</phase>
            <goals>
                <goal>read-project-properties</goal>
            </goals>
        </execution>
        <!-- Load properties for the clean lifecycle -->
        <execution>
            <id>clean-lifecycle-properties</id>
            <phase>pre-clean</phase>
            <goals>
                <goal>read-project-properties</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Configuring the Copy Maven Plugin

We will use the Copy Maven plugin for two purposes:

  1. We copy the Solr configuration files to the correct directory when we start our Solr instance.
  2. We delete the Solr configuration files when we execute command mvn clean at command prompt.

We can get started by adding the following XML to the plugins section of our POM file:

<plugin>
    <groupId>com.github.goldin</groupId>
    <artifactId>copy-maven-plugin</artifactId>
    <version>0.2.5</version>
    <executions>
        <!-- Add executions here -->
    </executions>
</plugin>

Lets move on and find out how we can configure the Copy Maven plugin to copy and delete the Solr configuration files.

Copying Solr Configuration Files

We can copy the Solr configuration files by following these steps:

  1. Create an execution which runs the copy goal of Copy Maven plugin in the compile phase of the Maven default lifecycle.
  2. Copy the solr.xml file the home directory of our Solr instance. Ensure that the properties filtering is applied to file when it is copied.
  3. Copy the files found from the src/main/config directory to the solr.solr.home/solr.default.core.directory/conf directory.
  4. Copy the language specific configuration files found from the src/main/config/lang directory to the solr.solr.home/solr.detault.core.directory/conf/lang directory.
  5. Copy the Velocity macros and other files found from the src/main/config/velocity directory to the solr.solr.home/solr.detault.core.directory/conf/velocity directory.
  6. Copy the XSL style sheets found from the src/main/config/xslt directory to the solr.solr.home/solr.detault.core.directory/conf/xslt directory.

The configuration of our execution looks as follows:

<execution>
    <id>copy-solr-config</id>
    <phase>compile</phase>
    <goals>
        <goal>copy</goal>
    </goals>
    <configuration>
        <resources>
            <!--
           Copy solr.xml to correct directory and applies properties
           filtering to it.
           -->
            <resource>
                <directory>${project.basedir}/src/main/resources</directory>
                <filtering>true</filtering>
                <targetPath>${solr.solr.home}</targetPath>
                <includes>
                    <include>solr.xml</include>
                </includes>
            </resource>
            <!-- Copy configuration files -->
            <resource>
                <directory>${project.basedir}/src/main/config</directory>
                <targetPath>${solr.solr.home}/${solr.default.core.directory}/conf</targetPath>
                <excludes>
                    <exclude>lang</exclude>
                    <exclude>velocity</exclude>
                    <exclude>xslt</exclude>
                </excludes>
            </resource>
            <!-- Copy language specific configuration files -->
            <resource>
                <directory>${project.basedir}/src/main/config/lang</directory>
                <targetPath>${solr.solr.home}/${solr.default.core.directory}/conf/lang</targetPath>
            </resource>
            <!-- Copy Velocity macros and other files -->
            <resource>
                <directory>${project.basedir}/src/main/config/velocity</directory>
                <targetPath>${solr.solr.home}/${solr.default.core.directory}/conf/velocity</targetPath>
            </resource>
            <!-- Copy XSL style sheets -->
            <resource>
                <directory>${project.basedir}/src/main/config/xslt</directory>
                <targetPath>${solr.solr.home}/${solr.default.core.directory}/conf/xslt</targetPath>
            </resource>
        </resources>
    </configuration>
</execution>

Deleting Solr Configuration Files

We can delete the Solr configuration files by following these steps:

  1. Create an execution which runs the copy goal of the Copy Maven plugin in the clean lifecycle phase.
  2. Ensure that build does not fail if the directories are not found.
  3. Delete the overlays directory which is created to the root directory of our Maven project.
  4. Delete the solr.xml file found from the home directory of our Solr instance.
  5. Delete the conf directory found from the solr.solr.home/solr.default.core.directory directory.

The configuration of our execution looks as follows:

<execution>
    <id>clean-solr</id>
    <phase>clean</phase>
    <goals>
        <goal>copy</goal>
    </goals>
    <configuration>
        <failIfNotFound>false</failIfNotFound>
        <resources>
            <!-- Clean the overlays directory from the project root directory -->
            <resource>
                <clean>true</clean>
                <cleanEmptyDirectories>true</cleanEmptyDirectories>
                <directory>${project.basedir}/overlays</directory>
                <includes>
                    <include>**/**</include>
                </includes>
            </resource>
            <!-- Remove the solr.xml file -->
            <resource>
                <clean>true</clean>
                <directory>${solr.solr.home}</directory>
                <includes>
                    <include>solr.xml</include>
                </includes>
            </resource>
            <!-- Remove the conf directory -->
            <resource>
                <clean>true</clean>
                <cleanEmptyDirectories>true</cleanEmptyDirectories>
                <directory>${solr.solr.home}/${solr.default.core.directory}</directory>
                <includes>
                    <include>conf</include>
                </includes>
            </resource>
        </resources>
    </configuration>
</execution>

Configuring the Jetty Maven Plugin

We can configure the Jetty Maven plugin to run our Solr instance by following these steps:

  1. Configure Jetty to listen the port 8983.
  2. Ensure that system properties are read from the profile specific configuration file. This property file contains a property called solr.solr.home which specifies the home directory of our Solr instance.
  3. Specify that the context path of our application is /solr.

The configuration of the Jetty Maven plugin looks as follows:

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>8.1.8.v20121106</version>
    <configuration>
        <stopPort>9966</stopPort>
        <stopKey>stop</stopKey>
        <connectors>
            <!-- Listen to port 8983 -->
            <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
                <port>8983</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <!-- Read system properties from profile specific configuration file -->
        <systemPropertiesFile>${project.basedir}/profiles/${build.profile.id}/config.properties</systemPropertiesFile>
        <webApp>
            <contextPath>/solr</contextPath>
        </webApp>
    </configuration>
</plugin>

Running Solr

We have now created a Maven build which can be used to run Solr in a development environment. We have got two options for starting our Solr instance:

  • We can execute mvn jetty:run command at command prompt.
  • We can execute mvn jetty:run-war command at command prompt.

After we have started Solr, we can access its admin interface by using the following url address: http://localhost:8983/solr.

The example application is available at Github. This example uses a custom schema because I plan to use it in my Spring Data Solr tutorial. The original example schema is found from the etc directory.

If you enjoy reading similar content, you should follow me on Twitter:

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He specializes in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

94 comments… add one

  • This is very useful.

    Reply
    • Thank you. It is always nice to get positive feedback.

      Reply
  • Thanks for a great blog post.

    What’s the value of the solr.solr.home property supposed to be?

    Reply
    • Thank you for your comment.

      The value of the solr.solr.home property configures the home directory of the used Solr instance. In other words, it configures the directory in which the Solr configuration file (solr.xml) and the core specific configuration files are copied when the compile phase of the Maven default lifecycle is executed.

      You can set it to point to the directory of your choosing. I will clarify the section which describes the usage of this property to ensure that this is explained properly.

      Reply
  • Thanks for a very informative and useful post. I tried adding another core dynamically and it threw this error
    org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
    ….
    Caused by: java.io.IOException: Can’t find resource ‘solrconfig.xml’ in classpath or ‘c:Temp\new_core\conf/’, cwd=C:\Documents\workspace-sts-2.8.1.RELEASE\solr

    Any idea why its not able to find solrconfig.xml and where is it looking for it.

    Reply
    • The problem is that Solr cannot find the solrconfig.xml file of your new core. The idea behind multiple cores is that each Solr core has its own configuration, indexes and schema. If you want to get more information about this, check out this wiki page.

      You can modify the example project to run a Solr instance which has multiple cores by making the following changes to it:

      1. Add the new core to the solr.xml file.
      2. Create a separate configuration files for each core.
      3. Modify the pom.xml file so that these files are copied to the correct directories when the project is compiled.
      4. Modify the pom.xml file so that these files are removed when the mvn clean command is executed.

      I hope that this answered to your question.

      Reply
      • Thanks for your response. Yeah I understand how cores are supposed to be setup but I was hoping that once solr is running it should be able to create new cores based on existing cores configuration files. I mean thats what the admin cores functionality is supposed to provide, the ability to manage cores dynamically. I understand what you are suggesting which is to replicate everything for another core which makes sense but I was hoping to not have to do that. Thanks though.

        Reply
        • I think that I misunderstood your original comment. I thought that you want to modify the example project to support multiple Solr cores. Sorry about that.

          It should be possible to create new cores based on the configuration of an existing core. You can do this by using the CREATE action of the CoreAdminHandler.

          Reply
          • yeah sorry I was not creating it the right away. I havent tried this dynamic creation of cores before and still now sure how to make it work. It seems like I am supposed to specify the instanceDir name of an existing core to create a new one but then it specifies the same instance dir for both cores which isnt right. Anyways the errors is gone so I now I just have to figure out how the dynamic creation of cores work which is more of a solr question. Again thanks for a very useful post.

      • Petri

        Thanks a ton for these valuable posts about solr.

        I am trying implement solr for my own ecom site. I was just trying out the example, jetty server is up, but http://localhost:8983/solr/ url gives me the following exception. Tried out couple of things, but no clue.. Can you please help me out?
        “SolrCore Initialization Failures
        liqon: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load config for solrconfig.xml”

        “There are no SolrCores running.
        Using the Solr Admin UI currently requires at least one SolrCore.”

        Unable to load environment info from null/admin/system?wt=json.”

        Reply
        • The error message which you added to your comment does not tell much. It simply states that Solr could not started because it could not load the solrconfig.xml file.

          There can be several reasons for this. For example, you might have an error in the solrconfig.xml file or Solr cannot find the solrconfig.xml file from the classpath.

          The root cause of the problem is located at the end of the stack trace. Could you paste the entire stack trace to here or to Pastebin?

          Reply
          • Thanks a lot for your response.

            I have added following configuration in my solrconfig.xml:

            Still getting the same error…stack-trace as follows,
            ERROR – 2013-05-30 03:26:55.535; org.apache.solr.core.CoreContainer; Unable to create core: liquorsonline
            org.apache.solr.common.SolrException: The AdminHandler is not registered with the current core.
            at org.apache.solr.core.SolrCore.(SolrCore.java:821)
            at org.apache.solr.core.SolrCore.(SolrCore.java:618)
            at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
            at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
            at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
            at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            at java.util.concurrent.FutureTask.run(FutureTask.java:138)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            at java.util.concurrent.FutureTask.run(FutureTask.java:138)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
            at java.lang.Thread.run(Thread.java:662)
            Caused by: org.apache.solr.common.SolrException: The AdminHandler is not registered with the current core.
            at org.apache.solr.handler.admin.AdminHandlers.inform(AdminHandlers.java:70)
            at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:616)
            at org.apache.solr.core.SolrCore.(SolrCore.java:816)
            … 13 more
            Note: I removed the rest of the stack trace to make this comment more readable.

          • Have you configured any admin handlers in the solrconfig.xml file? The reason why I am is asking this is that the stack trace states that Solr cannot read the solrconfig.xml file because it cannot find the admin handler configuration.

            You can take a look at the solrconfig.xml file of my example application to see how you can configure admin handlers in the solrconfig.xml file (Search for a string ‘Admin Handlers’).

  • Petri,

    I followed the above steps and was able to bring up the application. However the logs indicate that its trying to create core called collection1 by default. Is there a configuration missing somewhere, where it cant find the solr.xml and so tries to create one by itself?

    Reply
    • If you take a look at the example application of this blog post, you notice that the solr.xml file is found from the src/main/resources directory (link).

      This file contains two properties which are configured in the profile specific configuration file. These properties are:

      • The solr.default.core.directory property configures the directory used to store the data and configuration of the Solr default core
      • The solr.default.core.name property configures the name of the Solr default core.

      The values of the properties are obtained by using properties filtering. The example application has only one profile which is called dev. Its configuration file is found the profiles/dev directory (link). As you can see, the config.properties file configures the directory and name of the Solr default core.

      When you run the example application, it will create a core called todo.

      I hope that this answered to your problem.

      Reply
      • I was using ${user.home}/blah/blah for solr.solr.home in the config.properties file. For some reason looks like it doesn’t like ${user.home} and so wasn’t able to locate the solr.xml file that did get copied to the right place. I had to provide the full path and then the app came up.

        Now the issue is that on the left hand pane of the app i do see all the links but right pane is blank with a spinner spinning endlessly. I would think I m still missing something as I m unable to see anything in the dashboard.

        Reply
        • Further is what i see in the firebug console upon loading the page;

          Error 404 Not Found

          HTTP ERROR: 404
          Problem accessing /solr$%7BadminPath%7D. Reason:
          Not Found
          Powered by Jetty://

          Reply
          • Does the directory ${solr.solr.home}/${solr.default.core.directory}/conf directory have velocity and xslt subdirectories?

            Also, does it contain the following files: admin-extra.html, admin-extra.menu-bottom.html and admin-extra.menu-top.html?

            These files are copied to this directory by using the Copy Maven plugin. This part of the process is described in the section called Configuring the Copy Maven Plugin.

  • Yes all the appropriate as described above have been copied to the right place by the maven-copy-plugin.

    Reply
    • I am starting to run out of ideas here. Does Jetty print anything to the console?

      By the way, which version of Solr are you using?

      Also, could you add the contents of the solr.xml file which is found from the ${solr.solr.home} directory here?

      Reply
      • I have pasted the console log at http://pastie.org/7341380. The version of solr is 4.1.0 as per your example.

        Reply
        • It seems that everything goes fine when Jetty is started.

          What happens when you access Solr admin UI with a web browser? Does it print anything to the console?

          Reply
          • Nothing gets printed to the console

  • One other thing, the WEB-INF is empty in my project as per your notes above. I m guessing its needed for the webapp to come as i do see the overlayed WEB-INF from the solr.war.

    Reply
    • Yes, the empty WEB-INF directory was needed so that Jetty would start.

      I remember having some problems with the user interface (some parts were blank) when I updated this blog post from Solr 4.0 to 4.1. However, I soon realized that the build script did not copy all required files to the conf directory.

      By the way, do you use relative path as the value of the solr.default.core.directory property?

      Reply
      • Yes.

        #SOLR PROPERTIES

        #Configures the directory used to store the data and configuration of the Solr default core
        solr.default.core.directory=ole

        #Configures the name of the Solr default core.
        solr.default.core.name=ole

        #SYSTEM PROPERTIES
        #Configures the home directory of Solr. Set the preferred directory path here.
        solr.solr.home=/Users/peris/kuali/main/dev/ole-contentstore/solr

        Reply
        • Also I noticed in your documentation that you were cleaning up the overlays folder under the project, but there is never one created. So I m guessing you might be missing something in the doc. Could you give it a try of setting up the project yourself and see what might be missing in the documentation above please?

          Reply
          • The overlays directory is created only if you run the project in IntelliJ Idea (The war depencency is unpacked to this directory). If you don’t use Idea, you should not never see that directory.

            I just checked the steps described in this blog posts and I could create a runnable Solr instance by following them. Is there any chance that I could see the source code of your Maven project and try to figure out what is wrong?

        • Note that you can override the properties on the command line:

          mvn jetty:run -Dsolr.solr.home=/Users/darkblue/solr_mvn_3

          Reply
  • I have uploaded my project to https://github.com/perivenkat/ole-solr-sandbox.git
    Let me know if you run into issues. I use IntelliJ 12

    Reply
    • I found a mistake from the solr.xml file. You are using the solr element inside the solr element. When I removed the redundant solr element from this file, the Solr admin UI started to work.

      Reply
  • I also noticed that the data directory didn’t get created under my solr.home or under the core when the app came up.

    Reply
    • Yes. This was caused by the mistake which I found from the solr.xml file.

      Reply
      • Petri,

        Thanks for figuring it out. that did it. Really appreciate it.

        Reply
        • You are welcome!

          Reply
  • I m trying to embed solr in my java webapp; You think its a viable approach or should I go with the embeddedsolrserver?

    Reply
  • This approach works well with jetty but have you had the chance to figure this out with tomcat. It seems like the whole concept with profiles simply wont work with maven if the same war file needs to be created and deployed on different envs of tomcat e.g. dev and prod and if I want to use different configuration folder for each env. Have you had the chance to look into something like this

    Reply
    • Good idea. I will add this one on my to-do list as well.

      Reply
  • This is very useful, but not 4.3(solr).
    do you 4.3 demo? thanks.

    Reply
    • I am planning to update this blog post when I have got time to it. I will add this on my to-do list.

      Reply
  • Petri,

    Creating new thread.

    The following entry
    is there for Admin Handlers since beginning. One more thing I forgot to mention I am using Solr-4.3.0, if I start the jetty server with “java -jar start.jar” from example directory, that works fine.
    Now, if I start from “mvn jetty:run” [shown like in this example], then only I get the exception.

    Reply
    • It seems that the Wordpress “ate” the snippet which you added to your comment. I hate that feature but I guess it is for my own protection.

      Anyway, I have not tested the example with Solr 4.3.0. I guess I will do that in the weekend and make the necessary modifications to it (and to this blog post).

      I will let you know how it goes.

      Reply
      • Petri

        I was trying out with solr 4.3.0 and the examples you posted ..got the following error while running “mvn clean install”..
        Failed to execute goal on project liq-server: Could not resolve dependencies for project liquormart:liquormart-server:jar:1.0-SNAPSHOT: Could not find artifact org.apache.solr:solr:jar:4.3.0 in spring-milestone (http://repo.springsource.org/libs-milestone)

        I am not sure whether spring-milestone has support for solr 4.3.0….I have tried to download solr 4.1 from apache solr site.. but available stable versions are 3.6.2 and 4.3.0… didn’t get any link for solr 4.1.0 … so not sure what should I do now…
        Can you please guide me… whole solr implementation is stuck for my portal …
        Thanks in advance…

        Reply
        • I just updated the example application of this blog post to use Solr 4.3.0 and made the required changes to the blog post.

          The only change which I made was that I added the logging dependencies to the POM file. However, there were quite many changes in the files which I copied from the Solr binary distribution.

          You can see all changes by taking a look at this commit.

          About your first problem, since I did not encounter it during the update process, it is kind of hard to say what is wrong. You should to compare your solrconfig.xml with the solrconfig.xml of my example and add try to find a difference between them. If you cannot find one, you could create a new question to StackOverflow about it (to be honest, I am not really a Solr expert).

          About your second problem, Spring Data Solr 1.0.0.RC1 uses Solr 4.1.0. If you want to use Solr 4.3.0, you have two options:

          • Exclude the Solr related dependencies from the Spring Data Solr 1.0.0.RC1 dependency and declare them manually in the POM file.
          • Update your application to use Spring Data Solr 1.0.0.BUILD-SNAPSHOT. Remember to add the Spring snapshot repository to your POM file. The url of this repository is http://repo.springsource.org/libs-snapshot/.

          I hope that this answered to your question.

          Reply
          • Petri,

            Thanks a ton Mr. Petri. Sorry to trouble you again n again and late reply. I was little busy with another stuff. I got it up running now.

            My issue was.. initially I tried with another Solrj implementation, I had some unwanted dependency entries for lucene, those were causing problem. I have removed those dependency now solr is working fine. I will continue with your further solr implementations ….
            Thank you once again. Have a nice day ahead.

          • Don’t worry about the late reply.

            I am happy to hear that you found the root cause of your problem and fixed it.

  • Brilliant post. I was wrestling with this exact problem for hours today, your information got me up and running in no time – many thanks!

    Reply
    • You are welcome. It is nice to hear that I could help you out!

      Reply
  • Excellent explanation, this is the first time I see someone is explaining everything with more details, he cares about experts but never forget beginners like me, Thanks lot.

    Reply
  • I downloaded the source code, imported it in eclipse, the pom.xml has two errors :

    in this line :

    default-lifecycle-properties

    got this error :

    Plugin execution not covered by lifecycle configuration: org.codehaus.mojo:properties-
    maven-plugin:1.0-alpha-2:read-project-properties (execution: default-lifecycle-
    properties, phase: initialize)

    and in this line :

    copy-solr-config
    I got this one :
    Plugin execution not covered by lifecycle configuration: com.github.goldin:copy-
    maven-plugin:0.2.5:copy (execution: copy-solr-config, phase: compile)

    Thanks your help is appreciated.

    Reply
  • If you want to run this example in eclipse and to fix the above errors in pom.xml :

    That error is due to a missing tag. So, in order to avoid the exceptions in Eclipse, looks like one needs to simply enclose all the plugin tags inside a tag, like so:

    
    <build>
        <pluginManagement>
            <plugins>
                <plugin> ... </plugin>
                <plugin> ... </plugin>
                      ....
            </plugins>
        </pluginManagement>
    </build>
    
    

    Once this structure is in place, the error goes away.

    Reply
    • It seems that Wordpress decided to remove the XML tags from your response but I assume that you are referring to this StackOverflow answer. Am I correct? If this is the case, I can add the required configuration to your comment manually.

      Thanks for pointing out the solution to this problem.

      Reply
      • Yes Petri, I was referring to that link, it’s about adding pluginManagement tag.
        thanks lot

        Reply
        • I added the XML markup to your comment. This way you get the credit for solving this problem.

          Again, thank you for pointing out the solution for this problem.

          Reply
  • This project is using Jetty, How to make it work for tomcat in eclipse, because when I run it in eclipse I only got :

    http://localhost:8983/solr/

    http Status 404- /solr/
    description The requested resource is not available.

    Please your help is apopreciated.

    Reply
    • The main reason why I decided to use the Maven Jetty plugin was that this project was meant to be used during the development phase.

      Although it is of course handy to use an IDE for editing the configuration files, I think that running the project by using an IDE is less handy (Although this can be done by executing either the run or run-war goals of the Maven Jetty plugin).

      Do you want to deploy the war file to an existing Tomcat instance?

      Reply
      • Thanks again Petri, I am always learning from tutorials by running them in eclipse with Tomcat, if there is a way someone could show us how to run it in:
        1) eclipse with tomcat
        OR
        2)eclipse with Jetty

        thanks lot.

        Reply
        • The easiest way is to invoke the run goal of the Maven Jetty plugin by using M2E. I found one tutorial which describes how you can do this.

          I have also been thinking that I should write a blog post which describes how you can deploy this project into a Tomcat. In other words, this tutorial would describe how you can update the production Solr instance by using Maven. Maybe I will do that in the future.

          Reply
          • Hi Petri,
            any updates on your plan to make your project work with “mvn tomcat6:run” ? I think it might be useful for me and others…
            Jan

          • Hi Jan,

            At the moment I am waiting that this issue is resolved (I have moved on to Maven 3.1.1). Let’s hope that it is resolved soon.

            By the way, would you be interested to read a blog post which describes how you can deploy Solr binary to a remove Tomcat server by using Maven?

  • solr.solr.home property still not clear, for example here is where I downloaded this project :

    Abdelmajids-iMac:running-solr-with-maven majid$ pwd
    /Users/majid/Documents/workspace/solr/running-solr-with-maven
    Abdelmajids-iMac:running-solr-with-maven majid$ ls
    LICENSE etc profiles target
    README pom.xml src
    Abdelmajids-iMac:running-solr-with-maven majid$

    what should I set :
    solr.solr.home= ?????????

    Thanks lot, please bear with me if I ask lot of questions,

    Reply
    • Don’t worry about asking too many questions. We all do it when we are learning new technologies or tools.

      The solr.solr.home property has been described with more details in one of my earlier comment.

      I hope that it answers to your question.

      Reply
      • Thanks,
        I have already read that comment, you said :
        You can set it to point to the directory of your choosing.

        Can I just create a directory somewhere and setsolr.solr.home to that directory ?
        thanks

        Reply
        • Yes.

          Reply
  • I am using solr 1.4. Can we use for this solr version.

    Reply
    • I have confess that I am not sure if this is possible. If it is possible to run Solr 1.4 in a servlet container, it should be possible.

      However, you might have to adapt these instructions because I assume that it doesn’t have the same files than Solr 4.3.0.

      Reply
  • Hey Petri,
    that’s awesome and super-detailed, thank you so much! I’m happy, I learned something new ;-)

    It worked like a charm for me and even the upgrade to SOLR 4.5.0 was easy.

    I’d need an option to start jetty on a specific port, rather then 8983 – probably not a huge deal, just have to digg a bit.

    I’ve got a web app which uses SOLR as NoSQL storage for biological data which I plan to release soonish, with your tutorial users have an easy way to set up their own SOLR instance.

    again, great stuff man!!!! you’re awesome !

    cheers,
    Jan

    Reply
    • Hi Jan,

      I am happy to hear that this blog post was useful to you!

      By the way, you can specify the port also on the command line. For example, this command will start jetty in port 9999:

      mvn -Djetty.port=9999 jetty:run

      Reply
  • Just a heads up, the copy-maven-plugin is broken when using Maven 3.1.

    [WARNING] Error injecting: com.github.goldin.plugins.copy.CopyMojo
    java.lang.NoClassDefFoundError: Lorg/sonatype/aether/RepositorySystem;

    https://github.com/evgeny-goldin/maven-plugins/issues/10

    Thanks for the example though, was looking for an introduction to solr.

    Reply
    • Hi Steve,

      Thanks for the heads up! I will keep polling that issue and update this blog post after it has been resolved.

      Reply
    • I used ant plugin for copy. Worked more or less fine. Config sample – https://copy.com/4hn1PaEEksKj

      Reply
      • Thank you for the configuration sample. I will probably have to do something similar because it seems that the issue found from the copy-maven-plugin isn’t going to be fixed.

        Reply
        • I had impatience to wait when it gets resolved. I saw a few months history of people keeping asking..likely someone else should try solving it..but, well ant plugin with filtered copy works not badly so far for basic cases :).
          I have small issue with multicore / new style admin pages, but hoping to solve it soon.

          Reply
  • Hey Petri,
    Thanks for such a wonderful tutorial.You have explained every thing in details, and trust me man you have saved my whole day. Again Tons of thanks for such great tutorial.

    Reply
    • You are welcome! You just made my day.

      Reply
  • Nice post,
    The most interesting and complete I have ever seen.

    Reply
    • Thank you!

      Reply
  • I am not able to get the admin page. There is no errors, its just showing me the index page of jetty and not the solr admin page. I am using the same pom.xml as you so what could be the issue??..

    Thanks

    Reply
    • I ran into the same problem when I updated the Solr version of my example application. The root cause of my problem was that I hadn’t copied all the required files from the Solr distribution to my build.

      Check out the content of the following directories and ensure that you have copied all files:

      • example/solr/collection1/conf -> src/main/config
      • example/solr/collection1/conf/velocity -> src/main/config/velocity
      • example/solr/collection1/conf/xslt -> src/main/config/xslt

      What Solr version are you using?

      Reply
  • I’m trying do run this example , but is giving this error:
    {msg=SolrCore ‘collection1′ is not available due to init failure: Could not load config file /home/joao_paulo/NetBeansProjects/solr_install/collection1/solrconfig.xml,trace=org.apache.solr.common.SolrException: SolrCore ‘collection1′ is not available due to init failure: Could not load config file /home/joao_paulo/NetBeansProjects/solr_install/collection1/solrconfig.xml

    Caused by: java.io.IOException: Can’t find resource ‘solrconfig.xml’ in classpath or ‘/home/joao_paulo/NetBeansProjects/solr_install/collection1/conf/’, cwd=/home/joao_paulo/NetBeansProjects/solr

    the solr.solr.home folder still empty. no files inside.

    Reply
    • Which Maven version are you using?

      The Copy Maven plugin doesn’t work with Maven 3.1 yet. The author of that plugin has promised to release a new version of the plugin soon.

      Reply
      • maven 3.0.4
        java 1.7.0_25

        Reply
        • It seems that the problem is not your Maven version. I have some additional questions:

          • What is the value of the solr.solr.home property? Is it /home/joao_paulo/NetBeansProjects/solr_install/?
          • Are you running the example by using the command mvn jetty:run?
          • Did you download the example application from Github or did you create it manually?

          To be honest, this problem seems pretty weird. Is it possible to see your version of the project?

          Reply
      • Thanks for sharing this experience. All looks fine (I only see deprecation warnings in my solr 4.6.0). As temp workaround for copy and clean I used ant plugin in order to make it work with maven 3.1:

        org.apache.maven.plugins
        maven-antrun-plugin
        1.7

        copy-solr-config
        compile

        run

        clean-solr
        clean

        run

        In solr.xml:

        Reply
        • I think that Spring Data Solr 1.0 supports only Solr 4.3.1. You might be able to get rid of that deprecation warning by updating your Spring Data Solr version to 1.1.0.BUILD-SNAPSHOT. It seems to use Solr 4.6.1.

          By the way, the reason why I didn’t publish your second comment is that Wordpress removes the XML markup found from comments. :(

          Reply
          • Hi Petri,
            It was not not too much smart for me to forget sites don’t like markups. OK, no problem, I put sample pom and other configs here (hope, link will be remained alive) – https://copy.com/4hn1PaEEksKj
            I’m preparing multicore configs using solr 4.4+ dir structures (for core discovery approach). If somebody wants to see results, let me know. When I finish, I don’t know, so no promises with timelines :).

          • Hi Al,

            Great! Thanks for sharing!

  • Thanks Petri for this manual,that is what i was looking for.

    Regards

    Reply
  • As other said, thank you for this amazing post. Clever, simple and concise.
    Please, keep posting about solr and solr-spring-data.
    Really appreciated it.

    Reply
    • Thank you for your kind words!

      I have been planning to add a few new parts to my Spring Data Solr tutorial but I have been a bit lazy.

      Maybe it is time to see what is missing and finish the tutorial.

      Reply
  • Hi Petri,

    I wanted to utilize the dihs.jar which is the scheduler for the DIH and it seems that I need to add an applicationContextListener to my web.xml. From your tutorial, I understand that the maven war plugin overlays the solr.war and so I shouldn’t have my own defined web.xml. In such a case, how do I either merge the web.xml files (which I know is tricky) or enable the aplicaitonContextListener.

    The crude way would be to copy the solr’s web.xml into mine and add anything extra. Do you have any other alternatives?

    Thanks

    Reply
    • Are you trying to use the DIH scheduler in your Spring powered web application or add it to your Solr instance?

      As far as I am aware of, the DIH scheduler component is not supported by Solr yet (this seems to confirm my suspicions).

      I have to admit that I haven’t used the DIH scheduler myself but it seems that if you want to implement scheduled indexing, you have to to write some custom code and add this code to your web application or compile Solr and include this patch to your build.

      If you have some additional information about the DIH scheduler (especially about configuration), it would be great if you could share your information here.

      Reply

Leave a Comment