One common performance requirement for large websites is that different domain names should be used to serve static and dynamic content. The reason behind this requirement is that usually browsers do not download more than two components in parallel per domain name, because of the HTTP 1.1 specification. Thus, separating static and dynamic content under different hostnames is used to maximize the number of parallel downloads. More information about the performance increase obtained by the usage of this technique is given at Yahoo! User Interface Blog, which contains an entry titled Performance Research, Part 4: Maximizing Parallel Downloads in the Carpoon Line.
Recently I started wondering, how this requirement can be implemented by using Maven. The requirements for the Maven based solution are following:
- Static content of the Maven project must be automatically copied to the server used to server static content.
- It must be possible to copy CSS files and layout images to different remote directories.
- Created war file must be automatically copied to the server used to serve dynamic content.
- Files must be copied by using FTP protocol.
When I did some research, I learned that Maven Wagon plugin can be used to fulfill the given requirements.
Required Steps
The steps required to fulfill the given requirements are following:
- Enabling the FTP support of Maven Wagon Plugin
- Configuring the used credentials
- Configuring the Maven Wagon Plugin
Each of these steps are described with more details in following.
Enabling the FTP Support of Maven Wagon Plugin
However, since it does not support FTP protocol out of the box, I first tried to enable FTP support by following the instructions given in the usage page of the plugin's homepage. I soon noticed that this approach does not work, because the extension adding FTP support to Maven Wagon was not compatible with the commons-net-1.4.1.jar. So, instead of following the instructions given in the plugin's homepage, copy following jar files to your $MAVEN_HOME/lib directory:
- wagon-ftp-beta-6.jar
- oro-2.0.8.jar
- commons-net-2.0.jar
Configuring the Used Credentials
Before investigating the configuration added to the pom.xml file, we must configure the the credentials of the used servers to the settings.xml file. I recommend configuring the credentials by using user specific settings file (The default location of this file is ~/.m2/settings.xml). Following example contains own credentials for CSS files, layout images and the war file (Remember to replace the usernames and passwords with correct ones):
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <servers> <server> <id>css</id> <username>cssUser</username> <password>cssPassword</password> <!-- If you want deploy to active FTP server, uncomment the following block --> <!-- <configuration> <passiveMode>false</passiveMode> </configuration> --> </server> <server> <id>images</id> <username>imagesUser</username> <password>imagesPassword</password> <!-- If you want deploy to active FTP server, uncomment the following block --> <!-- <configuration> <passiveMode>false</passiveMode> </configuration> --> </server> <server> <id>war</id> <username>warUser</username> <password>warPassword</password> <!-- If you want deploy to active FTP server, uncomment the following block --> <!-- <configuration> <passiveMode>false</passiveMode> </configuration> --> </server> </servers> </settings>
Configuring the Maven Wagon Plugin
Now that we have fulfilled all the prerequisites for the usage of FTP via the Maven Wagon plugin, we can take a look of the pom file, which makes the magic happen. Since pom.xml is quite large, I will explain it in a smaller pieces, which should make it easier to understand. At first we will concentrate on the plugin declaration of the Maven Wagon plugin:
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>wagon-maven-plugin</artifactId> <version>1.0-beta-3</version> <!-- Contains unique execution declarations for CSS files, layout images and war file. --> <executions> <!-- Declaration for copying CSS files via FTP --> <execution> <id>upload-css</id> <!-- Describes the lifecycle phase when the plugin's goals are being executed. --> <phase>install</phase> <!-- Describes the goals are executed in the specified lifecycle phase. --> <goals> <goal>upload</goal> </goals> <configuration> <!-- Describes the local directory, which contents are copied. --> <fromDir>${css.local.dir}</fromDir> <!-- Describes the included files. --> <includes>*</includes> <!-- Describes the url of the remote FTP server. --> <url>${css.remote.url}</url> <!-- Describes the remote directory, where the contents of the local directory are copied. --> <toDir>${css.remote.dir}</toDir> <!-- Describes the server id, which is used to fetch the used credentials from settings.xml. --> <serverId>${css.remote.server.id}</serverId> </configuration> </execution> <!-- Declaration for copying images via FTP. See the declaration of CSS files for more information about the configuration. --> <execution> <id>upload-images</id> <phase>install</phase> <goals> <goal>upload</goal> </goals> <configuration> <fromDir>${images.local.dir}</fromDir> <includes>*</includes> <url>${images.remote.url}</url> <toDir>${images.remote.dir}</toDir> <serverId>${images.remote.server.id}</serverId> </configuration> </execution> <!-- Declaration for copying war file via FTP. See the declaration of CSS for more information about the configuration. --> <execution> <id>upload-war</id> <phase>install</phase> <goals> <goal>upload</goal> </goals> <configuration> <fromDir>${war.local.dir}</fromDir> <includes>*</includes> <url>${war.remote.url}</url> <toDir>${war.remote.dir}</toDir> <serverId>${war.remote.server.id}</serverId> </configuration> </execution> <executions> </plugin>
Cool. We have now configured the Maven Wagon plugin to copy CSS files, images and war file to remote FTP server when install lifecycle phase is executed. Obviously we now have to also configure the Maven War plugin to exclude CSS files and images from the war file. This can be done by using following plugin declaration:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>2.0.2</version> <configuration> <!-- Excludes the contents of the styles and image directories when copying files to the directory used to create war file. Note that the directories are hard coded and you might have to change the configuration depending on your project structure. --> <warSourceExcludes> styles/*, images/* </warSourceExcludes> </configuration> </plugin>
The last interesting part of the pom file contains the property declarations used in the configuration of Maven Wagon plugin. These declarations are shown in following:
<properties> <!-- Settings for CSS files --> <css.local.dir>${basedir}/src/main/webapp/styles</css.local.dir> <!-- Insert the remote target directory path for CSS files --> <css.remote.dir></css.remote.dir> <css.remote.server.id>css</css.remote.server.id> <!-- Insert the url address of the FTP server for CSS files --> <css.remote.url></css.remote.url> <!-- Settings for image files --> <images.local.dir>${basedir}/src/main/webapp/images</images.local.dir> <!-- Insert the remote target directory path for images --> <images.remote.dir></images.remote.dir> <images.remote.server.id>images</images.remote.server.id> <!-- Insert the url address of the FTP server for image files --> <images.remote.url></images.remote.url> <!-- Settings for War files--> <war.local.dir>${basedir}/target</war.local.dir> <!-- Insert the remote target directory path for war file --> <war.remote.dir></war.remote.dir> <war.remote.server.id>war</war.remote.server.id> <!-- Insert the url address of FTP server for war file --> <war.remote.url></war.remote.url> </properties>
The deployment process can be triggered by executing command: mvn install in the project's root directory (The directory, which contains the pom.xml file). The outcome is following:
- CSS files are copied from the configured local directory to the specified remote directory.
- Images are copied from the configured local directory to the specified remote directory.
- War file is constructed, and copied from the configured local directory to the specified remote directory.
Final Words
It should now be clear, how the Maven Wagon plugin can be used to copy static and dynamic files to different hosts. However, the current configuration is far from perfect. To be honest, it is not ready for production usage. Different Maven profiles for development, testing and production usage should be added, before I would recommend the usage of this configuration in serious development. The master plan behind this is following:
At this point the only way to run the example project from working war file (styles and images are included in the war file), is to comment out the configuration section of the Maven War plugin declaration. The lack of profiles is also the reason explaining, why the upload goal of the Maven Wagon plugin is triggered by install lifecycle phase. When using different profiles, Maven could be configured in such manner that no files would be copied to remote servers and excluded from the war file, when the development profile is enabled. Also, separating test and production profiles would make it possible to copy files to different remote servers depending on the enabled profile. Of course this would also allow us to determine the url addresses, which are needed to access static content, based on the used profile.
In any case, this improvement is left as an exercise for the reader.
Disclaimer: The example project has been tested with JDK 1.6.0_20, Maven 2.2.0 and OS X 10.6.4. Browse the source code of the example project.
Instead of ~/m2/settings.xml you can use "mvn -s settings.xml" so specify own location. This is handy with CI server deploy tasks.
Jari, I totally forgoted the existence of that command line option. Nice tip.
Hi,
Very good! Very good indeed.
I just thought about this issue regarding our project this week. It's nice that you work during your holiday! :)
And that -s command line option is a nice thing to know.
Antti, since I no dot have got any time to play around with interesting challenges during my time at work, I decided to do it on own time... :)
Hi,
Thank for the article !
You can add recursive copy with just modify */**. It can be useful if you have a lot of subdirectories.
Henri,
it is nice to hear that this article was useful to you. Also, thanks for your tip concerning the recursive copy function.
Moved the example project to GitHub.
As the wagon-ftp set up so PassiveMode=false....
Thanks
Hi Domingo,
thanks for your comment. I will update the blog entry asap.
Hi Petri,
Thanks a lot for the explanation.
Just a small consideration:
It's not the first time I read about HTTP 1.1 standard limiting the number of parallel downloads.
However, the way I understand HTTP spec, it only says that you SHOULD not have more than two idle keep-alive connections against the same server. That is, it says nothing (IMHO) about the number of concurrent connections, as long as they are active. It only says you shouldn't keep alive more than two of them per site, when not in use.
Am I wrong? May be browsers are being too conservative?
Cheers!
Hi,
Thank you for asking this interesting question.
I actually picked that up from this Yuiblog article which says that:
At the time I didn't read the HTTP/1.1 spec but now that you mentioned it, I took a quick look at chapter 8 which says that:
In other words, that limit is just a recommendation. Then I did a quick Google search and found this StackOverflow question. I check the value of the
network.http.max-connections-per-server
config setting (I use Firefox 32.0.2) and noticed that the default setting is 6.So I guess the result is that the HTTP 1.1 specification gives a recommendation, and the browsers can either ignore it or follow it.
I have to admit that I haven't read the whole specification, but if the idea behind this limitation is to improve the response times, this limit probably means active connections. Right?
Recursive copy does not work for wagon-ftp. Endless loop:
Reply received: 250 Directory successfully changed.
Command sent: CWD .
..
Reply received: 250 Directory successfully changed.
Command sent: CWD .
I haven't used the Maven Wagon plugin in four years, and I cannot remember if I had some problems with recursive copy. Have you tried updating the Maven Wagon plugin to the latest version? Also, you might want to create a bug report.