Spring Data Solr Tutorial: Dynamic Queries

Solr is often referred as a search server which we can use when we are implementing full-text search functions. However, it is often wise to leverage the performance of Solr when we are implementing a search function which takes its input from a search form.

In this scenario, the executed search query depends from the received input. This means that the number of query parameters depends from the input entered to the search form. In other words, the executed search query is dynamic.

The previous part of my Spring Data Solr tutorial taught us how we can add custom methods to a single repository. It is time put this information in to use and find out how we can create dynamic queries with Spring Data Solr.

Let's get started.

Creating Dynamic Queries

This section describes how we can create dynamic queries with Spring Data Solr. It is divided into two subsections which are described in the following:

  • The first subsection describes the basics which we need to know before we can start working on the actual search function.
  • The second subsection describes how we can implement the search function of our example application by adding a custom method to our Spring Data Solr repository.

Learning the Basics

Before we can start implementing the search function of our example application, we need to know how we can create queries "manually" by using Spring Data Solr. We can create a query "manually" by following these steps:

  1. Create the search criteria.
  2. Create the query which holds the used search criteria.
  3. Execute the created query.

These steps are described with more details in the following.

Creating the Search Criteria

First, we have to create the search criteria for our query. We can do this by using the criteria classes which are described in the following:

Creating the Executed Query

Second, we have to create the executed query. The query classes of Spring Data Solr are described in the following:

Executing the Created Query

Third, we have to execute the created query. The SolrTemplate class implements several methods which we can use for this purpose. These methods are described in the following:

  • The long count(final SolrDataQuery query) method returns the number of documents found with the query given as a method parameter.
  • The UpdateResponse delete(SolrDataQuery query) method deletes the documents which match with the query given as a method parameter and returns an UpdateResponse object.
  • The T queryForObject(Query query, Class<T> clazz) method returns a single document which matches with the query given as a method parameter.
  • The FacetPage<T> queryForFacetPage(FacetQuery query, Class<T> clazz) method executes a facet query against Solr index and returns the query results as a FacetPage object.
  • The Page<T> queryForPage(Query query, Class<T> clazz) method executes the query against Solr index and returns the query results as an implementation of the Page interface.

Let’s move on and put this theory into practice.

Implementing the Search Function

The requirements of our search function are following:

  • The search function must return all todo entries which name or description contains some word of the given search term. In other words, if the search term is "Foo Bar", our search function must return todo entries which title or description contains either "Foo" or "Bar".
  • The search must be case insensitive.

Because our search function is not static, we have to create it by using a dynamic query. We can create dynamic queries with Spring Data Solr by adding custom method to our Spring Data Solr repository. In other words, we have to follow these steps:

  1. Create a custom interface which declares the added method.
  2. Implement the created interface.
  3. Modify the repository interface to extend the created interface.

These steps are described with more details in the following.

Creating the Custom Interface

First, we have to create a custom interface which declares our custom search method. We can do this by following these steps:

  1. Create an interface called CustomTodoDocumentRepository.
  2. Declare the search() method. This method takes the used search term as a method parameter and returns a list of TodoDocument objects.

The source code of the CustomTodoDocumentRepository interface looks as follows:

public interface CustomTodoDocumentRepository {

    public List<TodoDocument> search(String searchTerm);

	//Other methods are omitted.
}

Implementing the Created Interface

Second, we have to implement the custom interface which we created earlier. We can do this by following these steps:

  1. Create a class called TodoDocumentRepositoryImpl and implement the CustomTodoDocumentRepository interface.
  2. Annotate the class with the @Repository annotation.
  3. Add SolrTemplate field to the class and annotate it with the @Resource annotation.
  4. Implement the search() method.

The implementation of the search() method requires a more detailed description which is given here. We can implement the search() method by following these steps:

  1. Get the words of the search term.
  2. Construct the used search criteria by calling the private createSearchConditions() method and passing the words of the search term as a method parameter. This method creates the used search criteria by using the API of the Criteria class.
  3. Create the executed query by creating a new SimpleQuery object and pass the created Criteria object as a constructor parameter.
  4. Get the search results by calling the queryForPage() method of the SolrTemplate class. Pass the created query and the type of the expected result objects as method parameters.
  5. Return the search results by calling the getContent() method of the Page interface.

The source code of the TodoDocumentRepositoryImpl class looks as follows:

import org.springframework.data.domain.Page;
import org.springframework.data.solr.core.SolrTemplate;
import org.springframework.data.solr.core.query.Criteria;
import org.springframework.data.solr.core.query.SimpleQuery;
import org.springframework.stereotype.Repository;

import javax.annotation.Resource;

@Repository
public class TodoDocumentRepositoryImpl implements CustomTodoDocumentRepository {

    @Resource
    private SolrTemplate solrTemplate;

    @Override
    public List<TodoDocument> search(String searchTerm) {
        String[] words = searchTerm.split(" ");

        Criteria conditions = createSearchConditions(words);
        SimpleQuery search = new SimpleQuery(conditions);

        Page results = solrTemplate.queryForPage(search, TodoDocument.class);
        return results.getContent();
    }

    private Criteria createSearchConditions(String[] words) {
        Criteria conditions = null;

        for (String word: words) {
            if (conditions == null) {
                conditions = new Criteria("title").contains(word)
                        .or(new Criteria("description").contains(word));
            }
            else {
                conditions = conditions.or(new Criteria("title").contains(word))
                        .or(new Criteria("description").contains(word));
            }
        }

        return conditions;
    }

    //Other methods are omitted
}

Modifying the Repository Interface

Third, we have to make our custom search() method visible to the users of our repository. We can do this by extending the CustomTodoDocumentRepository interface. The source code of the TodoDocumentRepository interface looks as follows:

import org.springframework.data.solr.repository.SolrCrudRepository;

public interface TodoDocumentRepository extends CustomTodoDocumentRepository, SolrCrudRepository<TodoDocument, String> {

}

We have now added a custom search() method to our Spring Data Solr repository. Let's find out how we can use this method.

Using the Custom Method

We can use the custom method by modifying the search() method of the RepositoryTodoIndexService class. The new implementation of this method is very simple. It gets the search results by calling the search() method of our Spring Data Solr repository and returns the search results.

The source code of the RepositoryTodoIndexService class looks as follows:

import org.springframework.stereotype.Service;

import javax.annotation.Resource;
import java.util.List;

@Service
public class RepositoryTodoIndexService implements TodoIndexService {

    @Resource
    private TodoDocumentRepository repository;

	//Other methods are omitted.

    @Override
    public List<TodoDocument> search(String searchTerm) {
        return repository.search(searchTerm);
    }
}

Summary

We have now implemented a dynamic search function with Spring Data Solr. Although our search function was rather simple, we should now be able to implement more complex queries as well.

This tutorial has taught us two things:

  • We learned how we can create queries "manually" by using Spring Data Solr.
  • We learned that we have to implement dynamic search methods by adding custom method to a single repository.

The next part of my Spring Data Solr tutorial describes how we can sort our query results.

P.S. The example application of this blog post is available at Github.

If you want to learn how to use Spring Data Solr, you should read my Spring Data Solr tutorial.
32 comments… add one
  • Marek Chmiel Apr 8, 2013 @ 15:39

    Hi Petri!
    I would like to thank you for such a great tutorial! It is a really great introduction into spring data solr :)

    I followed your examples to develop my own application. I decided to use criterias to create dynamic queries. It works for simple queries but I have some problems with more complex ones. For example, I have to create the following solr query:

    fieldA:value1 AND (fieldB:value2 OR fieldC:value3)

    How to express this query using Criteria API correctly? I tried in many ways but I am not able to create query with parentheses (they are required in my query to ensure it's correctness).

    Kind regards,
    Marek

    • Petri Apr 8, 2013 @ 21:07

      Hi Marek!

      Thank you for your comment. It is nice to hear that you find this tutorial useful.

      I would build your query by using the following code:

      
      Criteria search = new Criteria("fieldA").is("value1")
                      .and(new Criteria("fieldB").is("value2")
                              .or(new Criteria("fieldC").is("value3")
                      ));
      
      

      Let me know if this works. I am kind of curious to know this because I have to admit that I have not used the Criteria API as much as I wanted to.

      • Sujay Bawaskar Oct 11, 2014 @ 6:58

        Hi Petri,

        This did not worked for me. Instead of that I had to use filter query.

        Thanks,
        Sujay

      • venkat b Nov 23, 2016 @ 7:16

        Everything you have explained properly but why did not explain how to run the project and see the result...please explain step by step..we have downloaded your project but do not know how to run the project.

        • Petri Nov 23, 2016 @ 16:51

          Hi,

          The README of the example application explains how you can run the example application. Note that if I remember correctly, the example application doesn't support Java 8. If you want to run it, you have to either to update the dependency versions or use Java 7.

  • salideng Nov 21, 2013 @ 4:12

    I use the query-methods ,but there is error:
    Caused by: org.apache.solr.common.SolrException: No such core:
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:112)
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
    at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
    at org.springframework.data.solr.core.SolrTemplate$3.doInSolr(SolrTemplate.java:136)
    at org.springframework.data.solr.core.SolrTemplate$3.doInSolr(SolrTemplate.java:128)
    at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:106)
    I don't know the problem.

    • Petri Nov 21, 2013 @ 20:04

      I would start by checking that the default core specified in the solr.xml is found from the schema.xml. For example, if your solr.xml looks as follows:

      
      <solr persistent="true">
      	<cores adminPath="/admin/cores" defaultCoreName="todo">
      		<core name="todo" instanceDir="todo" />
      	</cores>
      </solr>
      
      

      Your schema.xml should have the following schema element (the value of the name attribute is equal to the value of the defaultCoreName attribute):

      
      <schema name="todo" version="1.5">
      
      

      If this doesn't solve your problem, please let me know.

  • Jurio Dec 21, 2013 @ 15:38

    Hi Petri !
    Firstly, thank you to sharing all your knowledge about Spring data Solr, there are not much tutorial about spring data solr in the web.
    For my website i need to display facets and query facet results (like amazon website...)
    To do it with spring data solr,
    Firstly to get the list of facet,do i have to execute a facet request with method "queryForFacetPage"(with argument sometimes)
    Then to get the result, execute a query request with the method "queryForPage" (by specifying the arguments for reduce the results) ?

    Is there a method which do the twice in spring data solr api ?

    By the way, how can i use facet_queries in spring data solr ? for example get a range of value for a facet ?

    Thanks you !

    • Petri Dec 21, 2013 @ 18:50

      Hi Jurio,

      I have to confess that I haven't used the faceted search with Spring Data Solr. However, I found a nice Spring Data Solr tutorial which has a small section about faceting (search for a word faceting). Does this help?

      I might write a new Spring Data Solr tutorial about this if you think that it would be a good addition to my existing tutorials.

      • Jurio Dec 21, 2013 @ 19:14

        Thank you for your quick answer :)
        Yes iam sure that facet search is a good tutorial because facet is very useful for a website (get wise result by filtering...)
        By the way i found the solution to get the facets and results, it was easy...
        the method "queryForFacetPage" return a FacetPage which contains a field "content" to get the result.
        I think the class name "FacetPage" may be improve because it contains facets AND results and not only facets.

  • Hristijan Petreski Jan 30, 2014 @ 14:05

    Dear Petri,

    Great tutorials. Thank you so much. In the above example for complex queries:
    fieldA:value1 AND (fieldB:value2 OR fieldC:value3)

    what if value1 has white space? Lets say I want to search the sentence "social media" not "social"AND"media" but the sentence as a whole. Will escaping the white-space with '\' work?

    Thank you so much. Keep the good work.

    • Hristijan Petreski Jan 30, 2014 @ 15:50

      Never mind I just find that .is does this automatically. I was using .contains

      • Petri Jan 30, 2014 @ 22:06

        It is great to hear that you found a solution to your problem.

        I have to admit that I haven't implemented that kind of search condition yet. In other words, I learned something new as well. Thanks for coming back and posting the solution!

  • Johnny Nov 12, 2014 @ 9:30

    Hi Petri !
    Thank you for your tutorial!
    How can I define the TodoDocument.java if there are some dynamic fields in solr schema.xml?
    i.e. : *_s, *_i.
    Regards
    Johnny

    • Petri Nov 12, 2014 @ 19:51

      Hi Johnny,

      I have never used dynamic fields myself but I found this blog post that describes how you can define a dynamic field by using the @Field annotation. I hope that it answers to your question.

  • Dean Aug 25, 2015 @ 23:50

    Is the spring-data-solr project still an active project? I have not seen a lot of recent commits.

    • Petri Aug 26, 2015 @ 9:36

      I guess it is since it is an "official" Spring Data project. Of course I have no idea what kind of plans SpringSource has for it in the future.

  • akram Sep 22, 2015 @ 11:31

    Hello
    at first i want to thank you for this great tutorial

    i follow all the steps and i get a good working search feature but it seems like it's not case insensitive
    how did you do it ?

    Thanks

    • Petri Sep 22, 2015 @ 12:13

      Hi,

      It's true that created query is not case-insensitive. I have configured Solr to ensure that the search is case-insensitive.

      • akram Sep 22, 2015 @ 15:16

        Thank you for your quick response

        i have an other issue, i using the 'queryForPage' method to retrieve data :
        Page results = solrTemplate.queryForPage(searchQuery, Part.class);
        List list = results.getContent();
        and i have an other method to count the number of items:
        solrTemplate.count(searchQuery);
        the problem is that with the same searchQuery i get a list containing 10 items using the first method and i get 12 as a result of the count method
        Do you know how is possible ?

        • akram Sep 22, 2015 @ 15:50

          ok it seems clear that :
          solrTemplate.queryForPage(searchQuery, Part.class);
          List list = results.getContent();
          will retrieve only the 10 first items but how can i fetch all the items ?

          • Petri Sep 23, 2015 @ 22:11

            I have to confess that I have never run into this problem. I will take a look at this during the weekend, but you could try to update Spring Data Solr to the newest version, and see if you still run into the same problem.

          • Petri Oct 4, 2015 @ 17:47

            I was able to reproduce this issue. It seems that the SimpleQuery class defines a default page size (10) which is used if you don't specify your pagination options by invoking the setPageRequest() method.

            I wasn't able to find a clean solution to this problem, but you could follow these steps:

            1. Get the number of search results by invoking the count() method of the SolrTemplate class.
            2. Create a new Pageable object and set the number of search results as page size.
            3. Specify the pagination options by invoking the setPageRequest() method of the SimpleQuery class. Pass the created Pageable object as a method parameter.
          • Achint Kalra Feb 16, 2016 @ 15:15

            This is more of a solr configuration issue. You can change the default number of records returned by a query in solrconfig.xml requestHandler :

            explicit
            100

          • Petri Feb 16, 2016 @ 21:34

            Thank you for sharing that tip. I think that it is useful for other people who face the same problem.

  • Denden Sep 30, 2015 @ 13:41

    Hi Petri,
    Thanks for the tutorial.
    I would like to do something like: ( a OR b) AND (c OR d)
    can you help please?
    Thanks in advance.

    • Denden Sep 30, 2015 @ 14:17

      It's ok I found the answer I have to use the method connect() to create explicit bracketing thanks a lot for the tutorial it was very helpful.

      • Petri Sep 30, 2015 @ 17:38

        Hi,

        Thank you for your kind words. I really appreciate them! Also, it is good to hear that you were able to solve your problem.

      • Vesa Apr 20, 2016 @ 22:08

        Thank you so much, Denden!

        I was losing my mind trying to figure this out. I can't for the life of me figure out the interaction between Criteria and Crotch. It seems to violate every reasonable practice. Crotch extends Criteria, both are concrete classes, Criteria returns Crotch objects from Criteria#and() method, Crotch#and() overrides Criteria#and() and behaves entirely differently.

        Criteria#and() seems to make sense. If you "and" this node to another node, it creates a new and-node and sets this node and the other node as its children. It then replaces "this" node with the new node in the tree, maintaining a binary tree structure.

        Crotch#and() on the other hand just adds the new node to a flat list. It's not a binary tree. Because of this it easily loses precedence. This method makes absolutely no sense to me.

        Because of the great class structure, when you do "new Criteria().and(new Criteria()).and(new Criteria());" it actually first class Criteria#and() and then Crotch#and() and they actually behave differently. This is nuts.

        So bottom line: to achieve (a OR b) AND c, instead of a.or(b).and(c) you will have to call a.or(b).connect().and(c).

        • Konda Reddy Oct 4, 2019 @ 5:23

          Lol, you saved me today. I have spent almost fours hours debugging to figure out this issue but ended up finally googling which brought me here. I am lucky that I came across your comment.

  • ranjeet Apr 19, 2017 @ 1:32

    HI Dere,
    I am struggling to implement filters in spring solr. I have a requirement to apply filters based on selected fields on front end. e.g. model year or program ,both being multiple values on front end. So i need to mass modelyear ( yr1,yr2) or program (pgm1,pgm2). any of filters cud be selected or not selected. Can you please suggest sample implementation using spring boot and solr?

  • ajay jaiswal Jun 24, 2019 @ 14:41

    How can we fetch only a specific column (field) data using criteria and SimpleFacetQuery

    code snippet where i want to add this condtion.

    Criteria conditions = createSearchConditions(searchFields);
    conditions.and(new Criteria(SERVICE_IMAGE_ID_S_LOWER).isNotNull());
    SimpleFacetQuery query = new SimpleFacetQuery(conditions, new SolrPageRequest(page, size)).addSort(getSortOrder(sortBy, direction));

Leave a Reply