Solr is often referred as a search server which we can use when we are implementing full-text search functions. However, it is often wise to leverage the performance of Solr when we are implementing a search function which takes its input from a search form.
In this scenario, the executed search query depends from the received input. This means that the number of query parameters depends from the input entered to the search form. In other words, the executed search query is dynamic.
The previous part of my Spring Data Solr tutorial taught us how we can add custom methods to a single repository. It is time put this information in to use and find out how we can create dynamic queries with Spring Data Solr.
Let's get started.
Creating Dynamic Queries
This section describes how we can create dynamic queries with Spring Data Solr. It is divided into two subsections which are described in the following:
- The first subsection describes the basics which we need to know before we can start working on the actual search function.
- The second subsection describes how we can implement the search function of our example application by adding a custom method to our Spring Data Solr repository.
Learning the Basics
Before we can start implementing the search function of our example application, we need to know how we can create queries "manually" by using Spring Data Solr. We can create a query "manually" by following these steps:
- Create the search criteria.
- Create the query which holds the used search criteria.
- Execute the created query.
These steps are described with more details in the following.
Creating the Search Criteria
First, we have to create the search criteria for our query. We can do this by using the criteria classes which are described in the following:
- The org.springframework.data.solr.core.query.SimpleStringCriteria class is a basic criteria class which is used to specify the executed query by using already formatted query string. The query string specified in this class is executed as is. Thus, this class cannot be used to build dynamic queries.
- The org.springframework.data.solr.core.query.Criteria is a criteria class which is used to build dynamic queries. It has a fluent API which supports the chaining of multiple Criteria objects.
Creating the Executed Query
Second, we have to create the executed query. The query classes of Spring Data Solr are described in the following:
- The org.springframework.data.solr.core.query.SimpleQuery class is a query class which supports both pagination and grouping.
- The org.springframework.data.solr.core.query.SimpleFacetQuery class is a query class supporting faceted search.
- The org.springframework.data.solr.core.query.SimpleFilterQuery class is query class which supports filter queries.
Executing the Created Query
Third, we have to execute the created query. The SolrTemplate class implements several methods which we can use for this purpose. These methods are described in the following:
- The long count(final SolrDataQuery query) method returns the number of documents found with the query given as a method parameter.
- The UpdateResponse delete(SolrDataQuery query) method deletes the documents which match with the query given as a method parameter and returns an UpdateResponse object.
- The T queryForObject(Query query, Class<T> clazz) method returns a single document which matches with the query given as a method parameter.
- The FacetPage<T> queryForFacetPage(FacetQuery query, Class<T> clazz) method executes a facet query against Solr index and returns the query results as a FacetPage object.
- The Page<T> queryForPage(Query query, Class<T> clazz) method executes the query against Solr index and returns the query results as an implementation of the Page interface.
Let’s move on and put this theory into practice.
Implementing the Search Function
The requirements of our search function are following:
- The search function must return all todo entries which name or description contains some word of the given search term. In other words, if the search term is "Foo Bar", our search function must return todo entries which title or description contains either "Foo" or "Bar".
- The search must be case insensitive.
Because our search function is not static, we have to create it by using a dynamic query. We can create dynamic queries with Spring Data Solr by adding custom method to our Spring Data Solr repository. In other words, we have to follow these steps:
- Create a custom interface which declares the added method.
- Implement the created interface.
- Modify the repository interface to extend the created interface.
These steps are described with more details in the following.
Creating the Custom Interface
First, we have to create a custom interface which declares our custom search method. We can do this by following these steps:
- Create an interface called CustomTodoDocumentRepository.
- Declare the search() method. This method takes the used search term as a method parameter and returns a list of TodoDocument objects.
The source code of the CustomTodoDocumentRepository interface looks as follows:
public interface CustomTodoDocumentRepository { public List<TodoDocument> search(String searchTerm); //Other methods are omitted. }
Implementing the Created Interface
Second, we have to implement the custom interface which we created earlier. We can do this by following these steps:
- Create a class called TodoDocumentRepositoryImpl and implement the CustomTodoDocumentRepository interface.
- Annotate the class with the @Repository annotation.
- Add SolrTemplate field to the class and annotate it with the @Resource annotation.
- Implement the search() method.
The implementation of the search() method requires a more detailed description which is given here. We can implement the search() method by following these steps:
- Get the words of the search term.
- Construct the used search criteria by calling the private createSearchConditions() method and passing the words of the search term as a method parameter. This method creates the used search criteria by using the API of the Criteria class.
- Create the executed query by creating a new SimpleQuery object and pass the created Criteria object as a constructor parameter.
- Get the search results by calling the queryForPage() method of the SolrTemplate class. Pass the created query and the type of the expected result objects as method parameters.
- Return the search results by calling the getContent() method of the Page interface.
The source code of the TodoDocumentRepositoryImpl class looks as follows:
import org.springframework.data.domain.Page; import org.springframework.data.solr.core.SolrTemplate; import org.springframework.data.solr.core.query.Criteria; import org.springframework.data.solr.core.query.SimpleQuery; import org.springframework.stereotype.Repository; import javax.annotation.Resource; @Repository public class TodoDocumentRepositoryImpl implements CustomTodoDocumentRepository { @Resource private SolrTemplate solrTemplate; @Override public List<TodoDocument> search(String searchTerm) { String[] words = searchTerm.split(" "); Criteria conditions = createSearchConditions(words); SimpleQuery search = new SimpleQuery(conditions); Page results = solrTemplate.queryForPage(search, TodoDocument.class); return results.getContent(); } private Criteria createSearchConditions(String[] words) { Criteria conditions = null; for (String word: words) { if (conditions == null) { conditions = new Criteria("title").contains(word) .or(new Criteria("description").contains(word)); } else { conditions = conditions.or(new Criteria("title").contains(word)) .or(new Criteria("description").contains(word)); } } return conditions; } //Other methods are omitted }
Modifying the Repository Interface
Third, we have to make our custom search() method visible to the users of our repository. We can do this by extending the CustomTodoDocumentRepository interface. The source code of the TodoDocumentRepository interface looks as follows:
import org.springframework.data.solr.repository.SolrCrudRepository; public interface TodoDocumentRepository extends CustomTodoDocumentRepository, SolrCrudRepository<TodoDocument, String> { }
We have now added a custom search() method to our Spring Data Solr repository. Let's find out how we can use this method.
Using the Custom Method
We can use the custom method by modifying the search() method of the RepositoryTodoIndexService class. The new implementation of this method is very simple. It gets the search results by calling the search() method of our Spring Data Solr repository and returns the search results.
The source code of the RepositoryTodoIndexService class looks as follows:
import org.springframework.stereotype.Service; import javax.annotation.Resource; import java.util.List; @Service public class RepositoryTodoIndexService implements TodoIndexService { @Resource private TodoDocumentRepository repository; //Other methods are omitted. @Override public List<TodoDocument> search(String searchTerm) { return repository.search(searchTerm); } }
Summary
We have now implemented a dynamic search function with Spring Data Solr. Although our search function was rather simple, we should now be able to implement more complex queries as well.
This tutorial has taught us two things:
- We learned how we can create queries "manually" by using Spring Data Solr.
- We learned that we have to implement dynamic search methods by adding custom method to a single repository.
The next part of my Spring Data Solr tutorial describes how we can sort our query results.
P.S. The example application of this blog post is available at Github.
Hi Petri!
I would like to thank you for such a great tutorial! It is a really great introduction into spring data solr :)
I followed your examples to develop my own application. I decided to use criterias to create dynamic queries. It works for simple queries but I have some problems with more complex ones. For example, I have to create the following solr query:
fieldA:value1 AND (fieldB:value2 OR fieldC:value3)
How to express this query using Criteria API correctly? I tried in many ways but I am not able to create query with parentheses (they are required in my query to ensure it's correctness).
Kind regards,
Marek
Hi Marek!
Thank you for your comment. It is nice to hear that you find this tutorial useful.
I would build your query by using the following code:
Let me know if this works. I am kind of curious to know this because I have to admit that I have not used the Criteria API as much as I wanted to.
Hi Petri,
This did not worked for me. Instead of that I had to use filter query.
Thanks,
Sujay
Everything you have explained properly but why did not explain how to run the project and see the result...please explain step by step..we have downloaded your project but do not know how to run the project.
Hi,
The README of the example application explains how you can run the example application. Note that if I remember correctly, the example application doesn't support Java 8. If you want to run it, you have to either to update the dependency versions or use Java 7.
I use the query-methods ,but there is error:
Caused by: org.apache.solr.common.SolrException: No such core:
at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:112)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.springframework.data.solr.core.SolrTemplate$3.doInSolr(SolrTemplate.java:136)
at org.springframework.data.solr.core.SolrTemplate$3.doInSolr(SolrTemplate.java:128)
at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:106)
I don't know the problem.
I would start by checking that the default core specified in the solr.xml is found from the schema.xml. For example, if your solr.xml looks as follows:
Your schema.xml should have the following schema element (the value of the
name
attribute is equal to the value of thedefaultCoreName
attribute):If this doesn't solve your problem, please let me know.
Hi Petri !
Firstly, thank you to sharing all your knowledge about Spring data Solr, there are not much tutorial about spring data solr in the web.
For my website i need to display facets and query facet results (like amazon website...)
To do it with spring data solr,
Firstly to get the list of facet,do i have to execute a facet request with method "queryForFacetPage"(with argument sometimes)
Then to get the result, execute a query request with the method "queryForPage" (by specifying the arguments for reduce the results) ?
Is there a method which do the twice in spring data solr api ?
By the way, how can i use facet_queries in spring data solr ? for example get a range of value for a facet ?
Thanks you !
Hi Jurio,
I have to confess that I haven't used the faceted search with Spring Data Solr. However, I found a nice Spring Data Solr tutorial which has a small section about faceting (search for a word faceting). Does this help?
I might write a new Spring Data Solr tutorial about this if you think that it would be a good addition to my existing tutorials.
Thank you for your quick answer :)
Yes iam sure that facet search is a good tutorial because facet is very useful for a website (get wise result by filtering...)
By the way i found the solution to get the facets and results, it was easy...
the method "queryForFacetPage" return a FacetPage which contains a field "content" to get the result.
I think the class name "FacetPage" may be improve because it contains facets AND results and not only facets.
Dear Petri,
Great tutorials. Thank you so much. In the above example for complex queries:
fieldA:value1 AND (fieldB:value2 OR fieldC:value3)
what if value1 has white space? Lets say I want to search the sentence "social media" not "social"AND"media" but the sentence as a whole. Will escaping the white-space with '\' work?
Thank you so much. Keep the good work.
Never mind I just find that .is does this automatically. I was using .contains
It is great to hear that you found a solution to your problem.
I have to admit that I haven't implemented that kind of search condition yet. In other words, I learned something new as well. Thanks for coming back and posting the solution!
Hi Petri !
Thank you for your tutorial!
How can I define the TodoDocument.java if there are some dynamic fields in solr schema.xml?
i.e. : *_s, *_i.
Regards
Johnny
Hi Johnny,
I have never used dynamic fields myself but I found this blog post that describes how you can define a dynamic field by using the
@Field
annotation. I hope that it answers to your question.Is the spring-data-solr project still an active project? I have not seen a lot of recent commits.
I guess it is since it is an "official" Spring Data project. Of course I have no idea what kind of plans SpringSource has for it in the future.
Hello
at first i want to thank you for this great tutorial
i follow all the steps and i get a good working search feature but it seems like it's not case insensitive
how did you do it ?
Thanks
Hi,
It's true that created query is not case-insensitive. I have configured Solr to ensure that the search is case-insensitive.
Thank you for your quick response
i have an other issue, i using the 'queryForPage' method to retrieve data :
Page results = solrTemplate.queryForPage(searchQuery, Part.class);
List list = results.getContent();
and i have an other method to count the number of items:
solrTemplate.count(searchQuery);
the problem is that with the same searchQuery i get a list containing 10 items using the first method and i get 12 as a result of the count method
Do you know how is possible ?
ok it seems clear that :
solrTemplate.queryForPage(searchQuery, Part.class);
List list = results.getContent();
will retrieve only the 10 first items but how can i fetch all the items ?
I have to confess that I have never run into this problem. I will take a look at this during the weekend, but you could try to update Spring Data Solr to the newest version, and see if you still run into the same problem.
I was able to reproduce this issue. It seems that the
SimpleQuery
class defines a default page size (10) which is used if you don't specify your pagination options by invoking thesetPageRequest()
method.I wasn't able to find a clean solution to this problem, but you could follow these steps:
count()
method of theSolrTemplate
class.Pageable
object and set the number of search results as page size.setPageRequest()
method of theSimpleQuery
class. Pass the createdPageable
object as a method parameter.This is more of a solr configuration issue. You can change the default number of records returned by a query in solrconfig.xml requestHandler :
explicit
100
Thank you for sharing that tip. I think that it is useful for other people who face the same problem.
Hi Petri,
Thanks for the tutorial.
I would like to do something like: ( a OR b) AND (c OR d)
can you help please?
Thanks in advance.
It's ok I found the answer I have to use the method connect() to create explicit bracketing thanks a lot for the tutorial it was very helpful.
Hi,
Thank you for your kind words. I really appreciate them! Also, it is good to hear that you were able to solve your problem.
Thank you so much, Denden!
I was losing my mind trying to figure this out. I can't for the life of me figure out the interaction between Criteria and Crotch. It seems to violate every reasonable practice. Crotch extends Criteria, both are concrete classes, Criteria returns Crotch objects from Criteria#and() method, Crotch#and() overrides Criteria#and() and behaves entirely differently.
Criteria#and() seems to make sense. If you "and" this node to another node, it creates a new and-node and sets this node and the other node as its children. It then replaces "this" node with the new node in the tree, maintaining a binary tree structure.
Crotch#and() on the other hand just adds the new node to a flat list. It's not a binary tree. Because of this it easily loses precedence. This method makes absolutely no sense to me.
Because of the great class structure, when you do "new Criteria().and(new Criteria()).and(new Criteria());" it actually first class Criteria#and() and then Crotch#and() and they actually behave differently. This is nuts.
So bottom line: to achieve (a OR b) AND c, instead of a.or(b).and(c) you will have to call a.or(b).connect().and(c).
Lol, you saved me today. I have spent almost fours hours debugging to figure out this issue but ended up finally googling which brought me here. I am lucky that I came across your comment.
HI Dere,
I am struggling to implement filters in spring solr. I have a requirement to apply filters based on selected fields on front end. e.g. model year or program ,both being multiple values on front end. So i need to mass modelyear ( yr1,yr2) or program (pgm1,pgm2). any of filters cud be selected or not selected. Can you please suggest sample implementation using spring boot and solr?
How can we fetch only a specific column (field) data using criteria and SimpleFacetQuery
code snippet where i want to add this condtion.
Criteria conditions = createSearchConditions(searchFields);
conditions.and(new Criteria(SERVICE_IMAGE_ID_S_LOWER).isNotNull());
SimpleFacetQuery query = new SimpleFacetQuery(conditions, new SolrPageRequest(page, size)).addSort(getSortOrder(sortBy, direction));