Spring Data Solr Tutorial: CRUD (Almost)

In the previous part of my Spring Data Solr tutorial, we learned how we can configure Spring Data Solr. Now it is time to take a step forward and learn how we can manage the information stored in our Solr instance. This blog entry describes how we add new documents to the Solr index, update the information of existing documents and delete documents from the index.

We can make the necessary modifications to our example application by following these steps:

  1. Create a document class which contains the information stored in the Solr index.
  2. Create a repository interface for our Spring Data Solr repository.
  3. Create a service which uses the created repository.
  4. Use the created service.

These steps are described with more details in the following sections.

These blog entries provide additional information which helps us to understand the concepts described in this blog entry:

Creating the Document Class

The first step is to create a document class which contains the information added to the Solr index. A document class is basically just a POJO which is implemented by following these rules:

  • The @Field annotation is used to create a link between the fields of the POJO and the fields of the Solr document.
  • If the name of the bean’s field is not equal to the name of the document’s field, the name of the document’s field must be given as a value of the @Field annotation.
  • The @Field annotation can be applied either to a field or setter method.
  • Spring Data Solr assumes by default that the name of the document’s id field is 'id'. We can override this setting by annotating the id field with the @Id annotation.
  • Spring Data Solr (version 1.0.0.RC1) requires that the type of the document's id is String.

More information:

Let’s move on and create our document class.

In the first part of my Spring Data Solr tutorial, we learned that we have to store the id, description and title of each todo entry to the Solr index.

Thus, we can create a document class for todo entries by following these steps:

  1. Create a class called TodoDocument.
  2. Add the id field to the TodoDocument class and annotate the field with the @Field annotation. Annotate the field with the @Id annotation (This is not required since the name of the id field is 'id' but I wanted to demonstrate its usage here).
  3. Add the description field to the TodoDocument class and annotate this field with the @Field annotation.
  4. Add the title field to the TodoDocument and annotate this field with the @Field annotation.
  5. Create getter methods to the fields of the TodoDocument class.
  6. Create a static inner class which is used to build new TodoDocument objects.
  7. Add a static getBuilder() method to the TodoDocument class. The implementation of this method returns a new TodoDocument.Builder object.

The source code of the TodoDocument class looks as follows:

import org.apache.solr.client.solrj.beans.Field;
import org.springframework.data.annotation.Id;

public class TodoDocument {

    @Id
    @Field
    private String id;

    @Field
    private String description;

    @Field
    private String title;

    public TodoDocument() {

    }

    public static Builder getBuilder(Long id, String title) {
        return new Builder(id, title);
    }

	//Getters are omitted

    public static class Builder {
        private TodoDocument build;

        public Builder(Long id, String title) {
            build = new TodoDocument();
            build.id = id.toString();
            build.title = title;
        }

        public Builder description(String description) {
            build.description = description;
            return this;
        }

        public TodoDocument build() {
            return build;
        }
    }
}

Creating the Repository Interface

The base interface of Spring Data Solr repositories is the SolrCrudRepository<T, ID> interface and each repository interface must extend this interface.

When we extend the SolrCrudRepository<T, ID> interface, we must give two type parameters which are described in the following:

  • The T type parameter means the type of our document class.
  • The ID type parameter means the type of the document’s id. Spring Data Solr (version 1.0.0.RC1) requires that the id of a document is String.

We can create the repository interface by following these steps:

  1. Create an interface called TodoDocumentRepository.
  2. Extend the SolrCrudRepository interface and give the type of our document class and its id as type parameters.

The source code of the TodoDocumentRepository interface looks as follows:

import org.springframework.data.solr.repository.SolrCrudRepository;

public interface TodoDocumentRepository extends SolrCrudRepository<TodoDocument, String> {
}

Creating the Service

Our next step is to create the service which uses the created Solr repository. We can create this service by following these steps:

  1. Create a service interface.
  2. Implement the created interface.

These steps are described with more details in the following.

Creating the Service Interface

Our service interface declares two methods which are described in the following:

  • The void addToIndex(Todo todoEntry) method adds a todo entry to the index.
  • The void deleteFromIndex(Long id) method deletes a todo entry from the index.

Note: We can use the addToIndex() method for adding new todo entries to the Solr index and updating the information of existing todo entries. If an existing document has the same id than the new one, the old document is deleted and the information of the new document is saved to the Solr index (See SchemaXML @ Solr Wiki for more details).

The source code of the TodoIndexService interface looks as follows:

public interface TodoIndexService {

    public void addToIndex(Todo todoEntry);

    public void deleteFromIndex(Long id);
}

Implementing the Created Interface

We cam implement the service interface by following these steps:

  1. Create a skeleton implementation of our service class.
  2. Implement the method used to add documents to the Solr index.
  3. Implement the method used to delete documents from the Solr index.

These steps are described with more details in the following.

Creating a Skeleton Implementation of the Service Class

We can create a skeleton implementation of our service interface by following these steps:

  1. Create a class called RepositoryTodoIndexService and annotate this class with the @Service annotation. This annotation marks this class as a service and ensures that the class will be detected during the classpath scanning.
  2. Add a TodoDocumentRepository field to the RepositoryTodoIndexService class and annotate that field with the @Resource annotation. This annotation instructs the Spring IoC container to inject the actual repository implementation the service’s repository field.

The source code of our dummy service implementation looks as follows:

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;

@Service
public class RepositoryTodoIndexService implements TodoIndexService {

    @Resource
    private TodoDocumentRepository repository;

    //Add methods here
}

Adding Documents to the Solr Index

We can create the method which adds new documents to the Solr index by following these steps:

  1. Add the addToIndex() method to the RepositoryTodoIndexService class and annotate this method with the @Transactional annotation. This ensures that our Spring Data Solr repository will participate in Spring managed transactions.
  2. Create a new TodoDocument object by using the builder pattern. Set the id, title and description of the created document.
  3. Add the document to the Solr index by calling the save() method of the TodoDocumentRepository interface.

The source code of the created method looks as follows:

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;

@Service
public class RepositoryTodoIndexService implements TodoIndexService {

    @Resource
    private TodoDocumentRepository repository;

    @Transactional
    @Override
    public void addToIndex(Todo todoEntry) {
        TodoDocument document = TodoDocument.getBuilder(todoEntry.getId(), todoEntry.getTitle())
                .description(todoEntry.getDescription())
                .build();
        
		repository.save(document);
    }

	//Add deleteFromIndex() method here
}

Deleting Documents from the Solr Index

We can create a method which deletes documents from the Solr index by following these steps:

  1. Add the deleteFromIndex() method to the RepositoryTodoDocumentService class and annotate this method with the @Transactional annotation. This ensures that our Spring Data Solr repository will participate in Spring managed transactions.
  2. Delete document from the Solr index by calling the delete() method of the TodoDocumentRepository interface.

The source code of the created method looks as follows:

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;

@Service
public class RepositoryTodoIndexService implements TodoIndexService {

    @Resource
    private TodoDocumentRepository repository;

	//Add addToIndex() method here

    @Transactional
    @Override
    public void deleteFromIndex(Long id) {
        repository.delete(id.toString());
    }
}

Using the Created Service

Our last step is to use the service which we created earlier. We can do this by making the following modifications to the RepositoryTodoService class:

  1. Add the TodoIndexService field to the the RepositoryTodoService class and annotate this field with the @Resource annotation. This annotation instructs the Spring IoC container to inject the created RepositoryTodoIndexService object to the service’s indexService field.
  2. Call the addToIndex() method of the TodoIndexService interface in the add() method of the RepositoryTodoService class.
  3. Call the deleteFromIndex() method of the TodoIndexService interface in the deleteById() method of the RepositoryTodoService class.
  4. Call the addToIndex() method of the TodoIndexService interface in the update() method of the RepositoryTodoService class.

The source code of the RepositoryTodoService looks as follows:

import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;
import java.util.List;

@Service
public class RepositoryTodoService implements TodoService {

    @Resource
    private TodoIndexService indexService;

    @Resource
    private TodoRepository repository;

    @PreAuthorize("hasPermission('Todo', 'add')")
    @Transactional
    @Override
    public Todo add(TodoDTO added) {
        Todo model = Todo.getBuilder(added.getTitle())
                .description(added.getDescription())
                .build();

        Todo persisted = repository.save(model);
        indexService.addToIndex(persisted);

        return persisted;
    }

    @PreAuthorize("hasPermission('Todo', 'delete')")
    @Transactional(rollbackFor = {TodoNotFoundException.class})
    @Override
    public Todo deleteById(Long id) throws TodoNotFoundException {
        Todo deleted = findById(id);

        repository.delete(deleted);
        indexService.deleteFromIndex(id);

        return deleted;
    }

    @PreAuthorize("hasPermission('Todo', 'update')")
    @Transactional(rollbackFor = {TodoNotFoundException.class})
    @Override
    public Todo update(TodoDTO updated) throws TodoNotFoundException {
        Todo model = findById(updated.getId());

        model.update(updated.getDescription(), updated.getTitle());
        indexService.addToIndex(model);

        return model;
    }
}

Summary

We have successfully created an application which adds documents to the Solr index and deletes documents from it. This blog entry has taught us the following things:

  • We learned how we can create document classes.
  • We learned that we can create Spring Data Solr repositories by extending the SolrCrudRepository interface.
  • We learned that Spring Data Solr assumes by default that the name of the document’s id field is 'id'. However, we can override this setting by annotating the id field with the @Id annotation.
  • We learned that at the moment Spring Data Solr (version 1.0.0.RC1) expects that the id of a document is String.
  • We learned how we can add documents to the Solr index and delete documents from it.
  • We learned that Spring Data Solr repositories can participate in Spring managed transactions.

The next part of my Spring Data Solr tutorial describes how we can search information from the Solr index by using query methods.

P.S. The example application of this blog entry is available at Github.

If you want to learn how to use Spring Data Solr, you should read my Spring Data Solr tutorial.
32 comments… add one
  • Westt Mar 9, 2013 @ 13:08

    Hi, Petri
    When will next part of Spring Data Solr tutorial?

    • Petri Mar 9, 2013 @ 20:33

      I will start writing it tomorrow and publish it as soon as possible.

      • Westt Mar 9, 2013 @ 22:41

        Thanks, this is very useful tutorial.

        By the way, one of the last commits in Sprig Data Solr, removes limitation (only String) on type id field.

        "DATASOLR-48 - Add support for numeric document id " -
        https://github.com/SpringSource/spring-data-solr/commit/b9afaa58627390b69f95b6ef715f2f1f008cab1b

        • Petri Mar 10, 2013 @ 13:28

          I noticed that commit a bit earlier and it is indeed a useful fix that helps us to clean up the code a bit. I think that I will wait for the next release before updating this tutorial.

          Anyway, thanks for pointing this out here. It might be valuable to someone who wants to use the build snapshot in his project.

      • venkat b Nov 23, 2016 @ 9:06

        when i am running your project i am getting illegalstateexception failed to load applicationcontext and java.lang.IllegalStateException: Failed to load ApplicationContext
        at org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:157)
        at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:103)

        Update: I removed most of the stack trace - Petri

        • Petri Nov 23, 2016 @ 16:54

          Hi,

          Are you by any change using Java 8? The reason why I ask this is that the example application doesn't support Java 8 (its dependencies are too old). If you want to run it, you have to either update the dependencies manually or use Java 7.

  • bharat Aug 3, 2013 @ 22:50

    This documentation helped me lot.. Thank you..
    Is there anything for upload xml data files with different schema and tags into solr(i.e. indexing)?

    • Petri Aug 3, 2013 @ 23:53

      You are welcome!

      You can index XML documents by using the DataImportHandler or implementing a custom XML reader which adds new documents to the Solr index by using Spring Data Solr.

      If you have to index only a few different XML documents, you should probably implement your own indexer component. On the other hand, if you have to support multiple different XML documents, I would use the DataImportHandler.

      Check out this StackOverflow question for more details about this.

      I hope that this answered to your question.

      • bharat Aug 24, 2013 @ 7:24

        As you mentioned, using DataImportHandler(DIH) I created solr cores for import xml/csv files and now I am getting query results as I expect, thanks for your guidance.
        My application is combination of Spring 3.0 and Sencha Touch (ohh! mobile app), where the next step is to integrate it with solr so that we can provide search functionality on the documents which we uploaded through DIH.
        Can you please suggest the approach to integrate solr? Performance is one more my worry part in integration due to it is mobile app.

        • Petri Aug 24, 2013 @ 10:07

          I would probably create a REST API which provides the required search function to the client. I would also save the used search terms so that I know how users are using the API. This is helpful if the you encounter performance problems.

          Here are a few links which explains how you can tune the performance of Solr:

          Also, if the performance of a single Solr instance is not enough, you might have to use SolrCloud.

          I hope that this was helpful to you.

  • AP Aug 14, 2013 @ 20:01

    Hi Petri,
    I was trying out your examples (solr data series); it was a great help getting me going with spring-data-solr. I have a question related to solr date fields. How do i index the date fields? I tried modifying the solr schema.xml to add solr.TrieDateField as fieldType and added a field that used this. Then i tried passing a utc date string to this field. I am not able to get it to index though.

    Any ideas?

    • Petri Aug 14, 2013 @ 20:46

      Hi,

      First, you have declare the date field in the schema.xml file:

      
      <schema name="example" version="1.5">
          <fields>
      		<!-- Other fields are omitted -->
      		
      		<!-- The date field -->
      		<field name="creationDate" type="date" indexed="true" 
      		multiValued="false" omitNorms="true" stored="false" required="true"/>
      	</fields>
      	<types>
      		<!-- Specifies the date field type -->
      		<fieldType name="date" class="solr.DateField" sortMissingLast="true"/>
      	</types>
      </schema>
      
      

      Second, you have add the Date field to your document class:

      
      import java.util.Date;
      
      public class ExampleDocument {
      
      	//Other fields are omitted
      
      	@Field("creationDate")
      	private Date creationDate;
      
      	//Methods are omitted
      }
      
      

      This should do the trick!

      • AP Aug 14, 2013 @ 21:18

        That worked like a charm!

        Thanks

        • Petri Aug 14, 2013 @ 21:37

          You are welcome!

          • Ramon Jan 15, 2014 @ 16:00

            Is there any problem with tdate field?

            I get this error with latest snapshot.

            Failed to convert from type java.lang.String to type java.util.Date for value '1365517622000'

            Thanks.

          • Petri Jan 15, 2014 @ 20:15

            Are you talking about the 1.1.0.BUILD-SNAPSHOT? I haven't used it personally (I use 1.0.0.RELEASE in my projects) so I am not sure if the latest snapshot has problems with tdate fields.

            You might want to create a new issue to the Spring Data Solr issue tracker.

  • AP Aug 16, 2013 @ 18:09

    Hi Petri,

    I have one more complication in my spring data solr project. With your help I've been able to get everything indexed and working properly but i haven't been able to deal with a complex object scenario. I am not being able to index a list or a set inside my main class.In your todo example, I added a set of todopoints. Then i added a field called point of type text_general:

    I was expecting the RepositoryTodoIndexService.addToIndex to add the todopoints to my solr index but apparently that's not happening.

    Is there a different way of doing this?

    My todo document has a set :

    private Set todoPoints;

    And my TodoPoint is where I added the solr annotation:
    @Field("point")
    private String point;

    • AP Aug 16, 2013 @ 23:37

      Well, I figured it out!

      As solr does not seem to handle composite classes, I had to flatten the todoPoints Set to List todoPoints. And then had to annotate this List for solr indexing.

      @Field(“point”)
      List todoPoints;

      I had to then change my schema.xml to add the point field:

      Thanks

      • Petri Aug 17, 2013 @ 9:50

        Great! It is good to hear that you solved your problem.

        Also, I haven't found a way to use composite classes with Spring Data Solr. As far as I know, your solution (flattening the class hierarchy) is the only one.

  • Eugene Sep 8, 2013 @ 20:46

    Hi, Thank you for the tutorial.. it was really helpfull.

    I do however have a question about the use of a generic document object.. in Solj one can just use the SolrInputDocument object to load data into Solr. This does not require us to create a document object with present field values.. My use-case is to allow users to specify an arbitrary xml doc or database table (I have no idea what field names would be used) and have my application load this into solr. Is there a way in Spring Data to make use of the SolrInputDocument where I can specify the fields as key-value pairs?

    Thanks you.

    • Petri Sep 8, 2013 @ 21:15

      Actually this is possible. Depending from your use case, you might want to

      These tutorials explain how you can extend your Spring Data Solr repositories with custom methods. You can create the indexed SolrInputDocument object in your custom method and add it to Solr index by calling the saveDocument(SolrInputDocument doc) method of the SolrOperations interface.

      I hope that this answered to your question.

  • sumit Deshpande Oct 22, 2013 @ 9:11

    I followed your tutorial. It is very nice and helpful tutorial, But I am facing small problem in "interface TodoIndexService" here you mentioned a
    "void addToIndex(Todo todoEntry)" method adds a todo entry to the index. But Todo.java is missing

    • Petri Oct 22, 2013 @ 10:26

      I have left a lot of things out from this tutorial because I felt that were not related to Spring Data Solr. You can get the full source code of the example application from Github. Here is a direct link to the source code of the Todo class.

      I hope that this solves your problem.

  • jurio Oct 23, 2013 @ 23:56

    Thank your for your tutorial very well explained :)

    • Petri Oct 24, 2013 @ 0:07

      You are welcome!

  • rajasekar Oct 8, 2014 @ 8:54

    hi thank u for your tutorial very well explained :)
    i have a doubt.. if my document class having a field - which is list of another document object

    eg.
    class University{
    @Field
    List colleges;
    @Fileld
    List departments;
    }

    here College and Department are other document classes or simply other POJO classes.. how can i define type for these fields in schema.xml file

    Thanks in advance.

  • Ghislain Feb 3, 2016 @ 12:57

    Hi sir,

    Thanks for your post about Apache Solr.
    I'm based to your docs to create Solr Spring data project with Spring annotations, Apache Solr 4.9.1, JDK8, Cassandra 2.x(config is OK), multiple Documents: User, Group, Directory.

    I get following error :

    org.springframework.data.solr.UncategorizedSolrException: Document is missing mandatory uniqueKey field: id; nested exception is org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Document is missing mandatory uniqueKey field: id
    at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:145)
    at org.springframework.data.solr.core.SolrTemplate.saveBean(SolrTemplate.java:184)
    at org.springframework.data.solr.core.SolrTemplate.saveBean(SolrTemplate.java:178)
    at org.springframework.data.solr.repository.support.SimpleSolrRepository.save(SimpleSolrRepository.java:149)

    Repository:

    
    @Repository
    public interface AccountSolrRepository extends SolrCrudRepository {
    	
    	Optional findByCompanyName(String companyName);
    }
    
    

    Entity:

    
    @SolrDocument(solrCoreName = "collection1")
    @Table
    public class Account implements Serializable {
    	
    	@Id
    	@Field("account_id")
    	@Indexed
    	private String accountId;
    }
    
    

    Config:

    
    @Configuration
    @EnableSolrRepositories(
        basePackages = { "com.streamscale.repository.solr" }, 
        multicoreSupport = true
    )
    public class SolrConfig {
    
    	@Bean
    	public SolrServer solrServer(@Value("${solr.host}") String solrHost) {
    		return new HttpSolrServer(solrHost);
    	}
    }
    
    

    Update: I removed irrelevant code since the comment was unreadable - Petri

  • Vipul Vohra May 30, 2017 @ 12:41

    Hy Petri,
    I tried to implment your above tutorial, but wont be able to run succesfully. Actually i want to create core at run time, so whenever i use any repository method, then it gives an exception=======
    Expected mime type application/xml but got text/html=====

    => I am using solr template method to create solrTemplate with specific coreName
    SolrTemplate solrTemplate = new SolrTemplate(solrClient,coreName);
    new SolrRepositoryFactory(solrTemplate).getRepository(SolrRepository.class).save (toDoDocument);

    I have a bean (ToDoDocument) which i wanted to get indexed

    class ToDoDocument{
    @Field
    private String id;
    @Field
    private String name;}

    so whwenever I use save() then it gives an exception======= Expected mime type application/xml but got text/html=====

    => but whe i use single core and mention @solrDocument annotation over the ToDoDocument, then it works fine

    @SolrDocument(solrCoreName = "solrcore")
    public class ToDoDocument

    BUT, i dont want to hadcoded core name, so can you please give me any solution for my scenerio.

  • Mreetunjay Kumar Sep 26, 2017 @ 12:07

    Hi Petri,

    I am using Spring Data Solr 2.1.7 in one of my application and I am getting the following exception while saving the document in solr.

    org.springframework.data.solr.UncategorizedSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html.

    Error 404 Not Found

    HTTP ERROR 404
    Problem accessing /solr/productitem/update. Reason:
    Not FoundPowered by Jetty://

    I spoild my two days in resolving the issue but I failed and finally decided to email you regarding the same . Plz help me out.

    Thanks / Mreetunjay Kumar

    • Petri Sep 30, 2017 @ 10:08

      Hi,

      Is productItem the name of your collection? The reason why I ask this is that one possible reason for the 404 error is that the specified collection is not found. If this is the case, you are using the wrong URL.

Leave a Reply