Do want to get a better understanding of Spring web application architecture? If so, get started right now!

Running Solr with Maven

Search Bot

Solr is an open source search server that is built by using the indexing and search capabilities of Lucene Core, and it can be used for implementing scalable search engines with almost any programming language.

Even though Solr has many advantages, setting up a development environment is not one of them. This blog post fixes that problem.

We will learn how we can run Solr by using Maven and ensure that each developer uses the same configuration, schema, and Solr version.

Let’s start by getting the Solr configuration files.

Getting the Solr Configuration Files

The first thing that we have to do is to get the Solr configuration files and copy them to our project. We can do this by following these steps:

  1. Download the binary distribution of Solr 4.3.0.
  2. Extract the downloaded package to the desired directory.
  3. Go the root directory of the extracted binary distribution.
  4. Copy the following files from the directory example/solr/collection1/conf to the directory src/main/config: admin-extra.html, admin-extra-menu.menu-bottom.html, admin-extra.menu-top.html, currency.xml, elevate.xml, mapping-FoldToASCII.txt, mapping-ISOLatin1Accent.txt, protwords.xml, schema.xml, scripts.conf, solrconfig.xml, spellings.txt, stopwords.txt, synonyms.txt and update-script.js.
  5. Copy the language specific configuration files found from the directory example/solr/collection1/conf/lang to the directry src/main/config/lang.
  6. Copy the Velocity macros and other files found from the directory example/solr/collection1/conf/velocity to the directry src/main/config/velocity.
  7. Copy the XSL style sheets found from the directory example/solr/collection1/conf/xslt to the directry src/main/config/xslt.
  8. Copy the solr.xml file from the directory exaple/solr/collection1 to the directory src/main/resources.
  9. Create the directory src/main/webapp/WEB-INF. If we don’t create this directory, we cannot start our Solr instance.

Let’s move on and find out how we can configure our Maven build.

Configuring Our Maven Build

After we have copied the Solr configuration files to our project, we have to configure our Maven build. The requirements of our Maven build are:

  • The properties of our Maven build must be read from an external property file. The only exception to this rule is that the version numbers of the dependencies can be declared in our POM file.
  • The build process must copy the Solr configuration files to the configured directory when our Solr instance is started.
  • The build process must delete the home directory of our Solr instance when we execute the mvn clean command at command prompt.
  • It must be possible to start our Solr instance by using the Jetty Maven plugin.

We can create a Maven build that fulfils these requirements by following these steps:

  1. Create a POM file.
  2. Get the required dependencies.
  3. Create the properties file that contains the properties of our Maven build.
  4. Edit the solr.xml file.
  5. Read the property values from an external properties file.
  6. Copy the Solr configuration files to the correct directories.
  7. Clean the build.
  8. Configure the Jetty Maven plugin.

Let’s get started.

Creating the POM File

We have to create a basic POM file for a web application project. This POM file looks as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>net.petrikainulainen.maven</groupId>
    <artifactId>running-solr-with-maven</artifactId>
    <packaging>war</packaging>
    <version>0.1</version>
     
    <profiles>
        <!-- Add profile configuration here -->
    </profiles>
    <dependencies>
        <!-- Add dependencies here -->
    </dependencies>
    <build>
        <finalName>solr</finalName>
        <!-- Add filter configuration here -->
        <!-- Add resources configuration here -->
        <plugins>
            <!-- Add plugin configuration here -->
        </plugins>
    </build>
</project>

Getting the Required Dependencies

We can get the required dependencies by declaring the following dependencies in our pom.xml file:

  • SLF4J
  • SLF4J interceptors for the java.util.logging (JUL) and the java.commons.logging (JCL) logging frameworks.
  • SLF4J Log4J 1.2.X binding.
  • Log4J
  • Solr 4.3.0 (war)
We need to declare the logging dependencies in our POM file because the logging configuration of Solr was changed when Solr 4.3.0 was released.

The dependencies section of our POM file looks as follows:

<!-- SLF4J -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.7</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jcl-over-slf4j</artifactId>
    <version>1.7.7</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
    <version>1.7.7</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.7.7</version>
</dependency>
<!-- Log4j -->
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>
<!-- Solr 4.3.0 -->
<dependency>
    <groupId>org.apache.solr</groupId>
    <artifactId>solr</artifactId>
    <version>4.3.0</version>
    <type>war</type>
</dependency>

Creating the Properties File

Our next step is to create the properties file that is used in our Maven build, and add the required build profile configuration to our POM file.

First, we have to create the properties file which is used in our Maven build. We can do this by following these steps:

  1. Create the directory profiles/dev to the root directory of our Maven project.
  2. Create the properties file called config.properties to the profiles/dev directory.

Our properties file has three properties which are described in the following:

  • The solr.detault.core.directory property configures the value of the default core directory. This is a directory that is created under the home directory of our Solr instance. This directory has two subdirectories:
    • The conf directory contains the configuration of our Solr instance.
    • The data directory contains the Solr index.
  • The solr.default.core.name property configures the name of the default core.
  • The solr.solr.home property configures the home directory of our Solr installation. In other words, it configures the directory in which the Solr configuration file (solr.xml) and the core specific configuration files are copied when the compile phase of the Maven default lifecycle is invoked.

Our config.properties file looks as follows:

#SOLR PROPERTIES
#Configures the directory used to store the data and configuration of the Solr default core
solr.default.core.directory=todo
#Configures the name of the Solr default core.
solr.default.core.name=todo
 
#SYSTEM PROPERTIES
#Configures the home directory of Solr. Set the preferred directory path here.
solr.solr.home=

Second, we must configure the build profiles of our Maven build and use filtering to replace replace the variables included in our resources. We can do this by following these steps:

  1. Create a single profile called dev and ensure that it is the default profile of our build.
  2. Declare a property called build.profile.id and set its value to ‘dev’.
  3. Create a filter that reads the profile specific configuration file and replaces the variables found from our resources with the actual property values.

We can finish the steps one and two by adding the following profile configuration to the profiles section of our POM file:

<profile>
    <id>dev</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
        <build.profile.id>dev</build.profile.id>
    </properties>
</profile>

We can finish the step three by adding the following XML to the build section of our POM file:

<filters>
    <filter>${project.basedir}/profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
    <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Editing the solr.xml File

Because we configure the name and the instance directory of the Solr default core in our profile specific configuration file, we have to make some changes to the solr.xml file. We can make these changes by following these steps:

  1. Set the value of the defaultCoreName attribute of the cores element. Use the value of the solr.default.core.name property that is found from our profile specific properties file.
  2. Set the value of the name attribute of the core element. Use the value of the solr.default.core.name property that is found from our profile specific configuration file.
  3. Set the value of the instanceDir attribute of the core element. Use the value of the solr.default.core.directory property that is found from our profile specific configuration file.

The solr.xml file looks as follows (the relevant parts are highlighted):

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
  <cores adminPath="/admin/cores"
         defaultCoreName="${solr.default.core.name}"
         host="${host:}"
         hostPort="${jetty.port:}"
         hostContext="${hostContext:}"
         zkClientTimeout="${zkClientTimeout:15000}">
    <core name="${solr.default.core.name}" instanceDir="${solr.default.core.directory}" />
  </cores>
</solr>

Reading the Property Values From an External Properties File

Because we want that all property values used in our POM file are read from an external properties file, we have to use a plugin called the Properties Maven plugin. We can configure this plugin by following these steps:

  1. Ensure that the properties are read from the profile specific configuration file.
  2. Create an execution that runs the read-project-properties goal of the Properties Maven plugin in the initialize phase of the Maven default lifecycle.
  3. Create an execution that runs the read-project-properties goal of the Properties Maven plugin in the pre-clean phase of the Maven clean lifecycle.

The configuration of the Properties Maven plugin looks as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>properties-maven-plugin</artifactId>
    <version>1.0-alpha-2</version>
    <configuration>
        <files>
            <!-- Properties are read from profile specific property file -->
            <file>${project.basedir}/profiles/${build.profile.id}/config.properties</file>
        </files>
    </configuration>
    <executions>
        <!-- Load properties for the default lifecycle -->
        <execution>
            <id>default-lifecycle-properties</id>
            <phase>initialize</phase>
            <goals>
                <goal>read-project-properties</goal>
            </goals>
        </execution>
        <!-- Load properties for the clean lifecycle -->
        <execution>
            <id>clean-lifecycle-properties</id>
            <phase>pre-clean</phase>
            <goals>
                <goal>read-project-properties</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Copying the Solr Configuration Files to the Correct Directories

Our next step is to copy the Solr configuration files to the home directory of our Solr instance. We will use the Maven Resources plugin for this purpose.

The first thing that we have to do is to add the Maven Resources plugin to the plugins section of our pom.xml file. We can do this by adding the following snippet to the plugins section of our POM file:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-resources-plugin</artifactId>
	<version>2.7</version>
	<executions>
		<!-- Add executions here -->
	</executions>
</plugin>

Now we need to configure the executions that copies the Solr configuration files to the correct directories. We can do this by following these steps:

  1. We need to copy the solr.xml file found from the src/main/resources directory to the ${solr.solr.home} directory. This directory is the home directory of our Solr instance. Also, we need apply properties filtering to it because we want replace the placeholders found from that file with the property values found from our profile specific properties file.
  2. We need to copy the contents of the src/main/config directory to the ${solr.solr.home}/${solr.default.core.directory}/conf directory. This directory contains the configuration of our Solr instance’s default core.

First, we need to copy the solr.xml file to the home directory of our Solr instance. We can do this by following these steps:

  1. Create an execution that invokes the copy-resources goal of the Resources Maven plugin in the compile phase of the default lifecycle.
  2. Configure the execution to copy the solr.xml file found from the src/main/resources directory to the ${solr.solr.home} directory. Remember to apply properties filtering to the solr.xml file.

The configuration of our execution looks as follows:

<execution>
    <id>copy-solr-xml</id>
    <phase>compile</phase>
    <goals>
        <goal>copy-resources</goal>
    </goals>
    <configuration>
        <!-- Configure the directory in which the file is copied. -->
        <outputDirectory>${solr.solr.home}</outputDirectory>
        <resources>
            <!--
                Configure the copied file and apply properties filtering to it.
            -->
            <resource>
                <directory>${project.basedir}/src/main/resources</directory>
                <filtering>true</filtering>
                <includes>
                    <include>solr.xml</include>
                </includes>
            </resource>
        </resources>
    </configuration>
</execution>

Second, we need to copy the contents of the src/main/config directory to the ${solr.solr.home}/${solr.default.core.directory}/conf directory. We can do this by following these steps:

  1. Create an execution that invokes the copy-resources goal of the Resources Maven plugin in the compile phase of the default lifecycle.
  2. Configure the execution to copy contents of the src/main/config directory to the to the ${solr.solr.home}/${solr.default.core.directory}/conf directory.

The configuration of our execution looks as follows:

<execution>
    <id>copy-solr-config</id>
    <phase>compile</phase>
    <goals>
        <goal>copy-resources</goal>
    </goals>
    <configuration>
        <!-- Configure the target directory in which the files are copied. -->
        <outputDirectory>${solr.solr.home}/${solr.default.core.directory}/conf</outputDirectory>
        <!--
            Configure the directory which contents are copied to the target directory.
            Disable properties filtering.
        -->
        <resources>
            <resource>
                <directory>${project.basedir}/src/main/config</directory>
                <filtering>false</filtering>
            </resource>
        </resources>
    </configuration>
</execution>

Cleaning the Build

When we clean our build, we have to delete two directories that are described in the following:

  • We need to delete the home directory of our Solr instance.
  • We need to delete the overlays directory that is created to the root directory of our project when we start our Solr instance by using the Jetty Maven plugin.

We can configure the Maven Clean plugin to delete these directories. If we want to delete additional directories when the command mvn clean is invoked, we have to follow these steps:

  1. Add the fileSets element to the configuration of the Maven Clean plugin.
  2. Configure the deleted directories by adding fileSet elements inside the fileSets element. We can configure the path of the deleted directory by adding the directory element inside the fileSet element. We can use both relative and absolute paths. If we use relative paths, the starting point of that path is the root directory of our project.

If we want to delete the foo directory that is found from the root directory of our project, we have to use the following configuration:

<configuration>
	<filesets>
		<!-- Configure the deleted directory. -->
		<fileset>
			<directory>foo</directory>
		</fileset>
	</filesets>
</configuration>

The configuration of the Maven Clean plugin that deletes the target directory, the overlays directory, and the home directory of our Solr instance looks as follows:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-clean-plugin</artifactId>
	<version>2.6</version>
	<configuration>
		<filesets>
			<!-- Delete the overlays directory from the project root directory -->
			<fileset>
				<directory>overlays</directory>
			</fileset>
			<!-- Delete the Solr home directory -->
			<fileset>
				<directory>${solr.solr.home}</directory>
			</fileset>
		</filesets>
	</configuration>
</plugin>

Configuring the Jetty Maven Plugin

We will run our Solr by using the Jetty Maven plugin. We can configure it by following these steps:

  1. Configure Jetty to listen the port 8983.
  2. Ensure that system properties are read from the profile specific configuration file. This property file contains a property called solr.solr.home which specifies the home directory of our Solr instance. If this property is missing, we cannot start our Solr instance because Solr cannot find its configuration files.
  3. Specify that the context path of our application is /solr.

The configuration of the Jetty Maven plugin looks as follows:

<plugin>
	<groupId>org.eclipse.jetty</groupId>
	<artifactId>jetty-maven-plugin</artifactId>
	<version>9.2.3.v20140905</version>
	<configuration>
		<stopPort>9966</stopPort>
		<stopKey>stop</stopKey>
		<!-- Listen to port 8983 -->
		<httpConnector>
			<port>8983</port>
			<idleTimeout>60000</idleTimeout>
		</httpConnector>
		<!-- Read system properties from profile specific configuration file -->
		<systemPropertiesFile>${project.basedir}/profiles/${build.profile.id}/config.properties
</systemPropertiesFile>
		<!-- Set the context path -->
		<webApp>
			<contextPath>/solr</contextPath>
		</webApp>
	</configuration>
</plugin>

Running Solr

We have now created a Maven build that can be used for running Solr in a development environment. We have got two options for starting our Solr instance:

  • We can execute the mvn jetty:run command at command prompt.
  • We can execute the mvn jetty:run-war command at command prompt.

After we have started our Solr instance, we can access its admin interface by using the following url address: http://localhost:8983/solr.

If you want play around with the example application, you can get it from Github. This example uses a custom schema because I plan to use it in my Spring Data Solr tutorial. The original example schema is found from the etc directory.

If you want to learn more about Spring Data Solr, you should read my Spring Data Solr tutorial.

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

103 comments… add one

  • This is very useful.

    Reply
    • Thank you. It is always nice to get positive feedback.

      Reply
  • Thanks for a great blog post.

    What’s the value of the solr.solr.home property supposed to be?

    Reply
    • Thank you for your comment.

      The value of the solr.solr.home property configures the home directory of the used Solr instance. In other words, it configures the directory in which the Solr configuration file (solr.xml) and the core specific configuration files are copied when the compile phase of the Maven default lifecycle is executed.

      You can set it to point to the directory of your choosing. I will clarify the section which describes the usage of this property to ensure that this is explained properly.

      Reply
  • Thanks for a very informative and useful post. I tried adding another core dynamically and it threw this error
    org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
    ….
    Caused by: java.io.IOException: Can’t find resource ‘solrconfig.xml’ in classpath or ‘c:Temp\new_core\conf/’, cwd=C:\Documents\workspace-sts-2.8.1.RELEASE\solr

    Any idea why its not able to find solrconfig.xml and where is it looking for it.

    Reply
    • The problem is that Solr cannot find the solrconfig.xml file of your new core. The idea behind multiple cores is that each Solr core has its own configuration, indexes and schema. If you want to get more information about this, check out this wiki page.

      You can modify the example project to run a Solr instance which has multiple cores by making the following changes to it:

      1. Add the new core to the solr.xml file.
      2. Create a separate configuration files for each core.
      3. Modify the pom.xml file so that these files are copied to the correct directories when the project is compiled.
      4. Modify the pom.xml file so that these files are removed when the mvn clean command is executed.

      I hope that this answered to your question.

      Reply
      • Thanks for your response. Yeah I understand how cores are supposed to be setup but I was hoping that once solr is running it should be able to create new cores based on existing cores configuration files. I mean thats what the admin cores functionality is supposed to provide, the ability to manage cores dynamically. I understand what you are suggesting which is to replicate everything for another core which makes sense but I was hoping to not have to do that. Thanks though.

        Reply
        • I think that I misunderstood your original comment. I thought that you want to modify the example project to support multiple Solr cores. Sorry about that.

          It should be possible to create new cores based on the configuration of an existing core. You can do this by using the CREATE action of the CoreAdminHandler.

          Reply
          • yeah sorry I was not creating it the right away. I havent tried this dynamic creation of cores before and still now sure how to make it work. It seems like I am supposed to specify the instanceDir name of an existing core to create a new one but then it specifies the same instance dir for both cores which isnt right. Anyways the errors is gone so I now I just have to figure out how the dynamic creation of cores work which is more of a solr question. Again thanks for a very useful post.

      • Petri

        Thanks a ton for these valuable posts about solr.

        I am trying implement solr for my own ecom site. I was just trying out the example, jetty server is up, but http://localhost:8983/solr/ url gives me the following exception. Tried out couple of things, but no clue.. Can you please help me out?
        “SolrCore Initialization Failures
        liqon: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load config for solrconfig.xml”

        “There are no SolrCores running.
        Using the Solr Admin UI currently requires at least one SolrCore.”

        Unable to load environment info from null/admin/system?wt=json.”

        Reply
        • The error message which you added to your comment does not tell much. It simply states that Solr could not started because it could not load the solrconfig.xml file.

          There can be several reasons for this. For example, you might have an error in the solrconfig.xml file or Solr cannot find the solrconfig.xml file from the classpath.

          The root cause of the problem is located at the end of the stack trace. Could you paste the entire stack trace to here or to Pastebin?

          Reply
          • Thanks a lot for your response.

            I have added following configuration in my solrconfig.xml:

            Still getting the same error…stack-trace as follows,
            ERROR – 2013-05-30 03:26:55.535; org.apache.solr.core.CoreContainer; Unable to create core: liquorsonline
            org.apache.solr.common.SolrException: The AdminHandler is not registered with the current core.
            at org.apache.solr.core.SolrCore.(SolrCore.java:821)
            at org.apache.solr.core.SolrCore.(SolrCore.java:618)
            at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
            at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
            at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
            at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            at java.util.concurrent.FutureTask.run(FutureTask.java:138)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            at java.util.concurrent.FutureTask.run(FutureTask.java:138)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
            at java.lang.Thread.run(Thread.java:662)
            Caused by: org.apache.solr.common.SolrException: The AdminHandler is not registered with the current core.
            at org.apache.solr.handler.admin.AdminHandlers.inform(AdminHandlers.java:70)
            at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:616)
            at org.apache.solr.core.SolrCore.(SolrCore.java:816)
            … 13 more
            Note: I removed the rest of the stack trace to make this comment more readable.

          • Have you configured any admin handlers in the solrconfig.xml file? The reason why I am is asking this is that the stack trace states that Solr cannot read the solrconfig.xml file because it cannot find the admin handler configuration.

            You can take a look at the solrconfig.xml file of my example application to see how you can configure admin handlers in the solrconfig.xml file (Search for a string ‘Admin Handlers’).

  • Petri,

    I followed the above steps and was able to bring up the application. However the logs indicate that its trying to create core called collection1 by default. Is there a configuration missing somewhere, where it cant find the solr.xml and so tries to create one by itself?

    Reply
    • If you take a look at the example application of this blog post, you notice that the solr.xml file is found from the src/main/resources directory (link).

      This file contains two properties which are configured in the profile specific configuration file. These properties are:

      • The solr.default.core.directory property configures the directory used to store the data and configuration of the Solr default core
      • The solr.default.core.name property configures the name of the Solr default core.

      The values of the properties are obtained by using properties filtering. The example application has only one profile which is called dev. Its configuration file is found the profiles/dev directory (link). As you can see, the config.properties file configures the directory and name of the Solr default core.

      When you run the example application, it will create a core called todo.

      I hope that this answered to your problem.

      Reply
      • I was using ${user.home}/blah/blah for solr.solr.home in the config.properties file. For some reason looks like it doesn’t like ${user.home} and so wasn’t able to locate the solr.xml file that did get copied to the right place. I had to provide the full path and then the app came up.

        Now the issue is that on the left hand pane of the app i do see all the links but right pane is blank with a spinner spinning endlessly. I would think I m still missing something as I m unable to see anything in the dashboard.

        Reply
        • Further is what i see in the firebug console upon loading the page;

          Error 404 Not Found

          HTTP ERROR: 404
          Problem accessing /solr$%7BadminPath%7D. Reason:
          Not Found
          Powered by Jetty://

          Reply
          • Does the directory ${solr.solr.home}/${solr.default.core.directory}/conf directory have velocity and xslt subdirectories?

            Also, does it contain the following files: admin-extra.html, admin-extra.menu-bottom.html and admin-extra.menu-top.html?

            These files are copied to this directory by using the Copy Maven plugin. This part of the process is described in the section called Configuring the Copy Maven Plugin.

  • Yes all the appropriate as described above have been copied to the right place by the maven-copy-plugin.

    Reply
    • I am starting to run out of ideas here. Does Jetty print anything to the console?

      By the way, which version of Solr are you using?

      Also, could you add the contents of the solr.xml file which is found from the ${solr.solr.home} directory here?

      Reply
      • I have pasted the console log at http://pastie.org/7341380. The version of solr is 4.1.0 as per your example.

        Reply
        • It seems that everything goes fine when Jetty is started.

          What happens when you access Solr admin UI with a web browser? Does it print anything to the console?

          Reply
          • Nothing gets printed to the console

  • One other thing, the WEB-INF is empty in my project as per your notes above. I m guessing its needed for the webapp to come as i do see the overlayed WEB-INF from the solr.war.

    Reply
    • Yes, the empty WEB-INF directory was needed so that Jetty would start.

      I remember having some problems with the user interface (some parts were blank) when I updated this blog post from Solr 4.0 to 4.1. However, I soon realized that the build script did not copy all required files to the conf directory.

      By the way, do you use relative path as the value of the solr.default.core.directory property?

      Reply
      • Yes.

        #SOLR PROPERTIES

        #Configures the directory used to store the data and configuration of the Solr default core
        solr.default.core.directory=ole

        #Configures the name of the Solr default core.
        solr.default.core.name=ole

        #SYSTEM PROPERTIES
        #Configures the home directory of Solr. Set the preferred directory path here.
        solr.solr.home=/Users/peris/kuali/main/dev/ole-contentstore/solr

        Reply
        • Also I noticed in your documentation that you were cleaning up the overlays folder under the project, but there is never one created. So I m guessing you might be missing something in the doc. Could you give it a try of setting up the project yourself and see what might be missing in the documentation above please?

          Reply
          • The overlays directory is created only if you run the project in IntelliJ Idea (The war depencency is unpacked to this directory). If you don’t use Idea, you should not never see that directory.

            I just checked the steps described in this blog posts and I could create a runnable Solr instance by following them. Is there any chance that I could see the source code of your Maven project and try to figure out what is wrong?

        • Note that you can override the properties on the command line:

          mvn jetty:run -Dsolr.solr.home=/Users/darkblue/solr_mvn_3

          Reply
  • I have uploaded my project to https://github.com/perivenkat/ole-solr-sandbox.git
    Let me know if you run into issues. I use IntelliJ 12

    Reply
    • I found a mistake from the solr.xml file. You are using the solr element inside the solr element. When I removed the redundant solr element from this file, the Solr admin UI started to work.

      Reply
  • I also noticed that the data directory didn’t get created under my solr.home or under the core when the app came up.

    Reply
    • Yes. This was caused by the mistake which I found from the solr.xml file.

      Reply
      • Petri,

        Thanks for figuring it out. that did it. Really appreciate it.

        Reply
        • You are welcome!

          Reply
  • I m trying to embed solr in my java webapp; You think its a viable approach or should I go with the embeddedsolrserver?

    Reply
  • This approach works well with jetty but have you had the chance to figure this out with tomcat. It seems like the whole concept with profiles simply wont work with maven if the same war file needs to be created and deployed on different envs of tomcat e.g. dev and prod and if I want to use different configuration folder for each env. Have you had the chance to look into something like this

    Reply
    • Good idea. I will add this one on my to-do list as well.

      Reply
  • This is very useful, but not 4.3(solr).
    do you 4.3 demo? thanks.

    Reply
    • I am planning to update this blog post when I have got time to it. I will add this on my to-do list.

      Reply
  • Petri,

    Creating new thread.

    The following entry
    is there for Admin Handlers since beginning. One more thing I forgot to mention I am using Solr-4.3.0, if I start the jetty server with “java -jar start.jar” from example directory, that works fine.
    Now, if I start from “mvn jetty:run” [shown like in this example], then only I get the exception.

    Reply
    • It seems that the Wordpress “ate” the snippet which you added to your comment. I hate that feature but I guess it is for my own protection.

      Anyway, I have not tested the example with Solr 4.3.0. I guess I will do that in the weekend and make the necessary modifications to it (and to this blog post).

      I will let you know how it goes.

      Reply
      • Petri

        I was trying out with solr 4.3.0 and the examples you posted ..got the following error while running “mvn clean install”..
        Failed to execute goal on project liq-server: Could not resolve dependencies for project liquormart:liquormart-server:jar:1.0-SNAPSHOT: Could not find artifact org.apache.solr:solr:jar:4.3.0 in spring-milestone (http://repo.springsource.org/libs-milestone)

        I am not sure whether spring-milestone has support for solr 4.3.0….I have tried to download solr 4.1 from apache solr site.. but available stable versions are 3.6.2 and 4.3.0… didn’t get any link for solr 4.1.0 … so not sure what should I do now…
        Can you please guide me… whole solr implementation is stuck for my portal …
        Thanks in advance…

        Reply
        • I just updated the example application of this blog post to use Solr 4.3.0 and made the required changes to the blog post.

          The only change which I made was that I added the logging dependencies to the POM file. However, there were quite many changes in the files which I copied from the Solr binary distribution.

          You can see all changes by taking a look at this commit.

          About your first problem, since I did not encounter it during the update process, it is kind of hard to say what is wrong. You should to compare your solrconfig.xml with the solrconfig.xml of my example and add try to find a difference between them. If you cannot find one, you could create a new question to StackOverflow about it (to be honest, I am not really a Solr expert).

          About your second problem, Spring Data Solr 1.0.0.RC1 uses Solr 4.1.0. If you want to use Solr 4.3.0, you have two options:

          • Exclude the Solr related dependencies from the Spring Data Solr 1.0.0.RC1 dependency and declare them manually in the POM file.
          • Update your application to use Spring Data Solr 1.0.0.BUILD-SNAPSHOT. Remember to add the Spring snapshot repository to your POM file. The url of this repository is http://repo.springsource.org/libs-snapshot/.

          I hope that this answered to your question.

          Reply
          • Petri,

            Thanks a ton Mr. Petri. Sorry to trouble you again n again and late reply. I was little busy with another stuff. I got it up running now.

            My issue was.. initially I tried with another Solrj implementation, I had some unwanted dependency entries for lucene, those were causing problem. I have removed those dependency now solr is working fine. I will continue with your further solr implementations ….
            Thank you once again. Have a nice day ahead.

          • Don’t worry about the late reply.

            I am happy to hear that you found the root cause of your problem and fixed it.

  • Brilliant post. I was wrestling with this exact problem for hours today, your information got me up and running in no time – many thanks!

    Reply
    • You are welcome. It is nice to hear that I could help you out!

      Reply
  • Excellent explanation, this is the first time I see someone is explaining everything with more details, he cares about experts but never forget beginners like me, Thanks lot.

    Reply
  • I downloaded the source code, imported it in eclipse, the pom.xml has two errors :

    in this line :

    default-lifecycle-properties

    got this error :

    Plugin execution not covered by lifecycle configuration: org.codehaus.mojo:properties-
    maven-plugin:1.0-alpha-2:read-project-properties (execution: default-lifecycle-
    properties, phase: initialize)

    and in this line :

    copy-solr-config
    I got this one :
    Plugin execution not covered by lifecycle configuration: com.github.goldin:copy-
    maven-plugin:0.2.5:copy (execution: copy-solr-config, phase: compile)

    Thanks your help is appreciated.

    Reply
  • If you want to run this example in eclipse and to fix the above errors in pom.xml :

    That error is due to a missing tag. So, in order to avoid the exceptions in Eclipse, looks like one needs to simply enclose all the plugin tags inside a tag, like so:

    
    <build>
        <pluginManagement>
            <plugins>
                <plugin> ... </plugin>
                <plugin> ... </plugin>
                      ....
            </plugins>
        </pluginManagement>
    </build>
    
    

    Once this structure is in place, the error goes away.

    Reply
    • It seems that Wordpress decided to remove the XML tags from your response but I assume that you are referring to this StackOverflow answer. Am I correct? If this is the case, I can add the required configuration to your comment manually.

      Thanks for pointing out the solution to this problem.

      Reply
      • Yes Petri, I was referring to that link, it’s about adding pluginManagement tag.
        thanks lot

        Reply
        • I added the XML markup to your comment. This way you get the credit for solving this problem.

          Again, thank you for pointing out the solution for this problem.

          Reply
  • This project is using Jetty, How to make it work for tomcat in eclipse, because when I run it in eclipse I only got :

    http://localhost:8983/solr/

    http Status 404- /solr/
    description The requested resource is not available.

    Please your help is apopreciated.

    Reply
    • The main reason why I decided to use the Maven Jetty plugin was that this project was meant to be used during the development phase.

      Although it is of course handy to use an IDE for editing the configuration files, I think that running the project by using an IDE is less handy (Although this can be done by executing either the run or run-war goals of the Maven Jetty plugin).

      Do you want to deploy the war file to an existing Tomcat instance?

      Reply
      • Thanks again Petri, I am always learning from tutorials by running them in eclipse with Tomcat, if there is a way someone could show us how to run it in:
        1) eclipse with tomcat
        OR
        2)eclipse with Jetty

        thanks lot.

        Reply
        • The easiest way is to invoke the run goal of the Maven Jetty plugin by using M2E. I found one tutorial which describes how you can do this.

          I have also been thinking that I should write a blog post which describes how you can deploy this project into a Tomcat. In other words, this tutorial would describe how you can update the production Solr instance by using Maven. Maybe I will do that in the future.

          Reply
          • Hi Petri,
            any updates on your plan to make your project work with “mvn tomcat6:run” ? I think it might be useful for me and others…
            Jan

          • Hi Jan,

            At the moment I am waiting that this issue is resolved (I have moved on to Maven 3.1.1). Let’s hope that it is resolved soon.

            By the way, would you be interested to read a blog post which describes how you can deploy Solr binary to a remove Tomcat server by using Maven?

  • solr.solr.home property still not clear, for example here is where I downloaded this project :

    Abdelmajids-iMac:running-solr-with-maven majid$ pwd
    /Users/majid/Documents/workspace/solr/running-solr-with-maven
    Abdelmajids-iMac:running-solr-with-maven majid$ ls
    LICENSE etc profiles target
    README pom.xml src
    Abdelmajids-iMac:running-solr-with-maven majid$

    what should I set :
    solr.solr.home= ?????????

    Thanks lot, please bear with me if I ask lot of questions,

    Reply
    • Don’t worry about asking too many questions. We all do it when we are learning new technologies or tools.

      The solr.solr.home property has been described with more details in one of my earlier comment.

      I hope that it answers to your question.

      Reply
      • Thanks,
        I have already read that comment, you said :
        You can set it to point to the directory of your choosing.

        Can I just create a directory somewhere and setsolr.solr.home to that directory ?
        thanks

        Reply
        • Yes.

          Reply
  • I am using solr 1.4. Can we use for this solr version.

    Reply
    • I have confess that I am not sure if this is possible. If it is possible to run Solr 1.4 in a servlet container, it should be possible.

      However, you might have to adapt these instructions because I assume that it doesn’t have the same files than Solr 4.3.0.

      Reply
  • Hey Petri,
    that’s awesome and super-detailed, thank you so much! I’m happy, I learned something new ;-)

    It worked like a charm for me and even the upgrade to SOLR 4.5.0 was easy.

    I’d need an option to start jetty on a specific port, rather then 8983 – probably not a huge deal, just have to digg a bit.

    I’ve got a web app which uses SOLR as NoSQL storage for biological data which I plan to release soonish, with your tutorial users have an easy way to set up their own SOLR instance.

    again, great stuff man!!!! you’re awesome !

    cheers,
    Jan

    Reply
    • Hi Jan,

      I am happy to hear that this blog post was useful to you!

      By the way, you can specify the port also on the command line. For example, this command will start jetty in port 9999:

      mvn -Djetty.port=9999 jetty:run

      Reply
  • Just a heads up, the copy-maven-plugin is broken when using Maven 3.1.

    [WARNING] Error injecting: com.github.goldin.plugins.copy.CopyMojo
    java.lang.NoClassDefFoundError: Lorg/sonatype/aether/RepositorySystem;

    https://github.com/evgeny-goldin/maven-plugins/issues/10

    Thanks for the example though, was looking for an introduction to solr.

    Reply
    • Hi Steve,

      Thanks for the heads up! I will keep polling that issue and update this blog post after it has been resolved.

      Reply
    • I used ant plugin for copy. Worked more or less fine. Config sample – https://copy.com/4hn1PaEEksKj

      Reply
      • Thank you for the configuration sample. I will probably have to do something similar because it seems that the issue found from the copy-maven-plugin isn’t going to be fixed.

        Reply
        • I had impatience to wait when it gets resolved. I saw a few months history of people keeping asking..likely someone else should try solving it..but, well ant plugin with filtered copy works not badly so far for basic cases :).
          I have small issue with multicore / new style admin pages, but hoping to solve it soon.

          Reply
  • Hey Petri,
    Thanks for such a wonderful tutorial.You have explained every thing in details, and trust me man you have saved my whole day. Again Tons of thanks for such great tutorial.

    Reply
    • You are welcome! You just made my day.

      Reply
  • Nice post,
    The most interesting and complete I have ever seen.

    Reply
    • Thank you!

      Reply
  • I am not able to get the admin page. There is no errors, its just showing me the index page of jetty and not the solr admin page. I am using the same pom.xml as you so what could be the issue??..

    Thanks

    Reply
    • I ran into the same problem when I updated the Solr version of my example application. The root cause of my problem was that I hadn’t copied all the required files from the Solr distribution to my build.

      Check out the content of the following directories and ensure that you have copied all files:

      • example/solr/collection1/conf -> src/main/config
      • example/solr/collection1/conf/velocity -> src/main/config/velocity
      • example/solr/collection1/conf/xslt -> src/main/config/xslt

      What Solr version are you using?

      Reply
  • I’m trying do run this example , but is giving this error:
    {msg=SolrCore ‘collection1′ is not available due to init failure: Could not load config file /home/joao_paulo/NetBeansProjects/solr_install/collection1/solrconfig.xml,trace=org.apache.solr.common.SolrException: SolrCore ‘collection1′ is not available due to init failure: Could not load config file /home/joao_paulo/NetBeansProjects/solr_install/collection1/solrconfig.xml

    Caused by: java.io.IOException: Can’t find resource ‘solrconfig.xml’ in classpath or ‘/home/joao_paulo/NetBeansProjects/solr_install/collection1/conf/’, cwd=/home/joao_paulo/NetBeansProjects/solr

    the solr.solr.home folder still empty. no files inside.

    Reply
    • Which Maven version are you using?

      The Copy Maven plugin doesn’t work with Maven 3.1 yet. The author of that plugin has promised to release a new version of the plugin soon.

      Reply
      • maven 3.0.4
        java 1.7.0_25

        Reply
        • It seems that the problem is not your Maven version. I have some additional questions:

          • What is the value of the solr.solr.home property? Is it /home/joao_paulo/NetBeansProjects/solr_install/?
          • Are you running the example by using the command mvn jetty:run?
          • Did you download the example application from Github or did you create it manually?

          To be honest, this problem seems pretty weird. Is it possible to see your version of the project?

          Reply
      • Thanks for sharing this experience. All looks fine (I only see deprecation warnings in my solr 4.6.0). As temp workaround for copy and clean I used ant plugin in order to make it work with maven 3.1:

        org.apache.maven.plugins
        maven-antrun-plugin
        1.7

        copy-solr-config
        compile

        run

        clean-solr
        clean

        run

        In solr.xml:

        Reply
        • I think that Spring Data Solr 1.0 supports only Solr 4.3.1. You might be able to get rid of that deprecation warning by updating your Spring Data Solr version to 1.1.0.BUILD-SNAPSHOT. It seems to use Solr 4.6.1.

          By the way, the reason why I didn’t publish your second comment is that Wordpress removes the XML markup found from comments. :(

          Reply
          • Hi Petri,
            It was not not too much smart for me to forget sites don’t like markups. OK, no problem, I put sample pom and other configs here (hope, link will be remained alive) – https://copy.com/4hn1PaEEksKj
            I’m preparing multicore configs using solr 4.4+ dir structures (for core discovery approach). If somebody wants to see results, let me know. When I finish, I don’t know, so no promises with timelines :).

          • Hi Al,

            Great! Thanks for sharing!

  • Thanks Petri for this manual,that is what i was looking for.

    Regards

    Reply
  • As other said, thank you for this amazing post. Clever, simple and concise.
    Please, keep posting about solr and solr-spring-data.
    Really appreciated it.

    Reply
    • Thank you for your kind words!

      I have been planning to add a few new parts to my Spring Data Solr tutorial but I have been a bit lazy.

      Maybe it is time to see what is missing and finish the tutorial.

      Reply
  • Hi Petri,

    I wanted to utilize the dihs.jar which is the scheduler for the DIH and it seems that I need to add an applicationContextListener to my web.xml. From your tutorial, I understand that the maven war plugin overlays the solr.war and so I shouldn’t have my own defined web.xml. In such a case, how do I either merge the web.xml files (which I know is tricky) or enable the aplicaitonContextListener.

    The crude way would be to copy the solr’s web.xml into mine and add anything extra. Do you have any other alternatives?

    Thanks

    Reply
    • Are you trying to use the DIH scheduler in your Spring powered web application or add it to your Solr instance?

      As far as I am aware of, the DIH scheduler component is not supported by Solr yet (this seems to confirm my suspicions).

      I have to admit that I haven’t used the DIH scheduler myself but it seems that if you want to implement scheduled indexing, you have to to write some custom code and add this code to your web application or compile Solr and include this patch to your build.

      If you have some additional information about the DIH scheduler (especially about configuration), it would be great if you could share your information here.

      Reply
  • Does this work within Eclipse? i.e. can I do the above configuration and run it from within Eclipse?

    Reply
  • Hi Petri

    Thank you for this blog post.
    Have followed your tutorial and have spent almost the whole day trying to figure out what could i have done wrong because am just not able to bring up jetty. Am using maven 3.0.5, solr 4.9.0. It just throws error 503 (service not available at localhost:8983/solr) and error 404 at localhost:8983. Jetty seems to come up fine in the console though. I would be glad if you could please provide me some direction/guidance. Am pretty new to solr and maven. Let me know if i could/should mail you my code. Please help.

    Many thanks!

    Reply
    • Do you see anything interesting in the log file when you get the 503 error?

      One possibility is that you haven’t copied all the required files from the example application. As you probably noticed, this blog post is based on Solr 4.3.0, but originally the example application used Solr 4.2.0 (I think). When I tried it with Solr 4.3.0, I couldn’t get it to work.

      Then I noticed that the directories mentioned in this blog post had a few new files. After I copied to these files to the correct directories, I was able to run my Solr instance by using the Jetty Maven plugin.

      Reply
  • Hi Petri, I’ve been following your guideline and all goes smoothly until I tried to access the REST API (https://wiki.apache.org/solr/SchemaRESTAPI) for configuring synonyms, stopwords, …

    When running with mvn jetty:run and accessing the following URL http://localhost:8080/solr/collection1/schema/analysis/stopwords/english , I get:

    “`
    2014-09-30 09:31:57.820:WARN:oejs.ServletHandler:Error for /solr/collection1/schema/fields
    java.lang.StackOverflowError
    at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304)
    at java.lang.StringCoding.encode(StringCoding.java:344)
    at java.lang.String.getBytes(String.java:916)
    at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
    at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
    at java.io.File.isDirectory(File.java:843)
    at org.eclipse.jetty.util.resource.FileResource.isDirectory(FileResource.java:240)
    at org.eclipse.jetty.util.resource.FileResource.addPath(FileResource.java:147)
    at org.eclipse.jetty.util.resource.ResourceCollection.addPath(ResourceCollection.java:207)
    at org.eclipse.jetty.server.handler.ContextHandler.getResource(ContextHandler.java:1567)
    at org.eclipse.jetty.webapp.WebAppContext.getResource(WebAppContext.java:358)
    at org.mortbay.jetty.plugin.JettyWebAppContext.getResource(JettyWebAppContext.java:324)
    at org.eclipse.jetty.server.handler.ContextHandler.getResourcePaths(ContextHandler.java:1622)
    at org.mortbay.jetty.plugin.JettyWebAppContext.getResourcePaths(JettyWebAppContext.java:397)
    at org.eclipse.jetty.server.handler.ContextHandler$Context.getResourcePaths(ContextHandler.java:1925)
    at org.restlet.ext.servlet.internal.ServletWarEntity.(ServletWarEntity.java:114)
    at org.restlet.ext.servlet.internal.ServletWarEntity.(ServletWarEntity.java:122)

    “`

    BTW, deploying the WAR generated by maven on Tomcat works well. So it’s not related to a missing dependency. Do you have any ideas of what’s wrong here ?

    Thanks.

    Reply
    • Hi Francois,

      Unfortunately I have no idea what could be wrong (I have never tried to access the REST API). That being said, you could:

      • Check the file permissions and ensure that the Jetty server has the right to modify the relevant files.
      • Update the build script to use the latest version of the Jetty Maven plugin.

      By the way, your question reminded me that I should update this blog post because this solution doesn’t work with Maven 3.1.X or Maven 3.2.X. I can take a look at your problem when I do that.

      Reply
  • Thank you Petri , It was nicely explained , I faced problem with the copy plugin on maven 3.2.1 but did a workaround using maven-resources-plugin to copy resources and maven-clean-plugin . This was really helpful. Thank you.

    Reply
    • Thank you for your kind words. I really appreciate them.

      Also, thank you for pointing out that you copied the files by using the maven-resources-plugin and cleaned them by using the maven-clean-plugin. I will update the blog post to use these plugins as well.

      Reply
      • Hi Petri,
        I read your other blogs regarding spring data solr .I have some confusion /question,when we can setup solr server standalone and access it via rest api .what is the use case for using spring-data solr to do the same .also why use embedded server and http server .
        If you can provide some use case and explain the need for spring-data-solr and different scenarios best suited for both server setups will be great.

        Thanks
        Piyush

        Reply

Leave a Comment