Posts

Friday Repost: Grails and Bamboo

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In our development environment, we use the complete Atlassian suite. We (or actually I) choose this environment because the products integrates so well together, but also because it integrates good with other products, like SpringSource Tool Suite, or, in our case, IntelliJ IDEA. And finally, the Atlassian suite has excellent support for building Grails projects, which, at the moment, is all we do. Simply put, we choose Atlassian because it rocks!

To integrate even better with the Atlassian Build Server (Bamboo), I’ve created a small script which will update the version number of the application with the Build Number. This way, we always know which build is running where. To do so, I created the following script.

This script will look for a file called ‘build-number.txt’, which is produced by the Build Number Stamper Plugin, and put in the working directory. This file is read by the script, and it updates the version number in application.properties, which will result in a war file produced by the build in the following format ‘..war’, for example ‘todolist.0.1.34.war’. This way you can easily relate each build which bugs you might find, and avoids confusion, for example between testers and developers.

As an extra, we’ve show the current version of the application in the source of our HTML. We do this by reading the version number from the application.properties with the following tag:

This will display the version number of the application, and will allow you to relate issues to a build number, to avoid miscommunication about the deployed application!

Friday Repost: Making Grails work behind an proxy server

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Making Grails work behind an NTLM firewall (or using Grails without Internet)

Currently, I’m in the process of taking over an existing Grails project and migrating it to a different location. This location has been setup by me, so I have full control over it, except for one tiny detail: the Proxy Server.

The application used to work great, and installing plugins was a breeze: just do a good old ‘grails install-plugin fitnesse’, and the plugin would be installed. However, since this Proxy Server is not just any Proxy Server, but a Microsoft NTLM Proxy Server, we tried quite hard to make Java work with it, but at the moment the score is 1-0 for the Proxy Server. This gives us the following problems:

1) Our dependencies cannot be downloaded
2) Our plugins cannot be be downloaded
3) Some plugins cannot be installed anymore

Downloading dependencies
In our BuildConfig.groovy, we specified some dependencies like joda time and xstream. These dependencies are normally downloaded, but due to our Proxy issue, that doesn’t work.

Our solution to this is to use Nexus. Nexus is able to pass the firewall, and can download our dependencies. It serves as a transparent Maven proxy, which can also be used by Ivy, and solves our problem nicely. Besides, it’s a good idea to install a local Maven proxy anyway, mostly because of reliability, but also because of speed and making sure everyone uses the same dependencies.

Downloading plugins
Unfortunately, the Grails plugins are not in a Maven repository. However, a solution to this is to download the plugin zip (eg grails-fitnesse-0.2.zip), rename it (to fitnesse-0.2.zip), put it in the lib directory (even though I’m really not a fan of putting binaries in version control systems), and register it in the application.properties and BuildConfig.groovy.

The BuildConfig.groovy should look like this:

As you can see, everything is configured here: the dependencies are here, the location to nexus is configured, and there’s a plugin closure. This “plugins” closure defines the plugins, and allows you to place them in the ‘lib’ directory. Be sure to rename the plugin zip to remove the ‘grails’ part, else it doesn’t work. Maybe these zips are also available from a Maven repository, which would solve the issue of putting binaries in our source control system, but I haven’t been able to find them yet.

Plugin installation
Some plugins, like the jQuery plugin, but also like my own plugin, the syntax-highlighter, download resources from the Internet when installing them. I haven’t been able to work around that yet, but my current solution is to just not use those plugins. I downloaded the jQuery javascript libraries and attached them to the project myself, and since we only use jQuery which does that, this solution is also an adequate fix.

Conclusion
So, as you can see, with some effort, it’s possible to make Grails work behind a Proxy Server, or without internet at all. This is a good solution for build servers, so you can have reproducible builds without the need for an Internet connection.

Friday Repost: Groovy Test Data Builder Pattern

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Most of us value the quality of our code highly. I do, and I expect you, as a reader, also to care for the quality of your code. Because of that, we write tests. Test which usually contain of 3 parts: the setup, the execution and the verification. And sometimes some cleanup. In this blog, I’d like to focus on the setup part, and try to get rid of some boilerplate code which can easily clutter our tests!

As I mentioned before, we usually write some setup code which serves as input for our test. This code can look like this:

Writing this code can be time consuming, plus the code doesn’t reveal the intention of the test. What are the significant bits in this setup code? Is it the age? Is it my name? Or a combination?

To solve the creation of testdata, there are multiple patterns. On of those patterns is the Test Data Builder pattern. The idea behind the Test Data Builder pattern is that test data should be easily to construct, which is usually accomplished by chaining method calls (also called a Fluent Interface, or a Builder).

I’ve taken the concept to Groovy and made an TestDataBuilder Category. A Groovy Category adds functionality to your code within a limited scope (defined by the ‘use’ keyword). This enables the object which is used in the setup of the test to ‘borrow’ the method of the TestDataBuilder Groovy class, which allows it to construct itself with testdata. This is best demonstrated by the use of an example:

By placing the creation of the data in the scope of the Category, the .build() method becomes available in for the Person class. Calling the build method will instantiate the Person class and populate all the fields with values which all equal true. So numbers will become 1, booleans true, Strings will get the name of the variable as a value, etc.

To use this in tests, and make the code more intention revealing, we can choose to populate some fields with our own values, and have the rest be generated.

The benefits of this is that you save a lot of setup code, and make the code a lot more clear and maintainable. If you want to use this, the code can be found on GitHub.com. (This class was previously called the ObjectMother, an error on my side. The change in name isn’t reflected yet in GitHub). Yes, there are some limitations, like it doesn’t handly all standard datatypes yet (like dates), nor does it populate associations (list with other objects) yet. Have fun, and any feedback on the code would be highly valued!

Friday Repost: Preventing spam with Groovy and Grails

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

When I created the jworks.nl website in Grails, one of my requirements was to have a blog. Since the website is created in Grails, and the Grails application doesn’t integrate easily with a standard blogging solution like WordPress, I decided to create my own blogging solution. But, since this proved to be much more painful than initially anticipated, I migrated the website to a new design and decided to use WordPress instead. However, the plugin creation was fun to do, so I’m sharing my experiences here.

The current blog consists of little more than just a BlogPost class, and a Comment class. This worked pretty well, and I got the blog up in no time. However, my blog was found by the spam bots, and it got spammed a little. In the first couple of days, I got 3 spam messages a day, which could be easily managed by hand (however still quite annoying). Then, the spam load increased to 10, 100 and then a 1000 per day. Then I got really annoyed, and decided it was time for a solution!

My solution was not to choose a Captcha. Captcha’s tend to bother the user, making it harder to post feedback. That’s not something I’d look forward to. So I created my own solution, based on how WordPress does it.

WordPress has their own spam blocking service, which is called Akismet. The nice thing about Akismet is that their service isn’t exclusive for WordPress, but they provide an API so others can also use it! Which is exactly what I did: I created a Grails plugin for Akismet, called (surprise surpise!): Grails Akismet! I’m sure you didn’t see that coming!

It’s very simple to use. Just do a ‘grails install akismet’, add your site plus the Aksimet key to your Grails configuration, and you’re good to go! The plugin provides a Grails Service, which can be injected into most Grails artifacts (like services, domains, and controllers). By using the ‘check for spam’ method in the AkismetService, a call to the online Akismet service is done, which determines if the comment is spam or not. Very easy, and very non-intrusive!

So, if you want to get started with the plugin, please check out the documentation and let me know what you think!

Friday Repost: Indexing documents with Tika and Grails

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In one of the previous blogposts, I described a way to easily upload images by using Grails commands. This blogpost further builds on that, by providing a generic way to index uploaded documents. These documents can later be retrieved by using the search option provided by the Grails Search plugin.

To index documents, we’re going to need two components: one of these components is the Grails Search plugin, as described above. The other component is Tika. Tika is a document indexer library which can index many kinds of documents like Text, Word, Excel, Powerpoint, but also more exotic document formats, like MP3 and FLV. Combined with Lucene, this provides a powerful combo.

So, how do we get this to work? Well, first we need the application from the previous blogpost. I refactored it a little to represent the new generic format better. I renamed all references from image to document (so now we have a CreateDocumentCommand instead of a CreateImageCommand, as well as all other references), and I moved the upload.gsp to the views directory, since redirection and forwarding works better that way. After the refactoring was done, I first installed the searchable plugin by typing:

grails install searchable

I also needed Tika, so I added it to the BuildConfig.groovy in the grails-app/conf directory, like this:

Grails and extra XML libraries really don’t mix well, so to prevent some nasty class loading issues, better exclude them here. What we need to do now is to tell searchable to index our documents when we upload them. This is quite trivial really, we can just annotate the Document class (which in the previous application was called ‘Image’) with a Tika converter. Then we have to create the converter, register it in Spring, and we should be done. So, first things first: let’s annotate the Document class.

In the Document class, add a searchable closure with a converter for the byte array, like this:

We now need to create the TikaConverter. We’ll create a TikaConverter.groovy in the src/groovy/com/example/converter directory. The class should look like this:

This class is responsible for extracting text from any kind of document uploaded to the system. To make this converter available, we have to register it in the Spring context, and then we’re ready to go. To register the Converter in the Spring context, add the following lines to grails-app/conf/spring/resources.groovy:

Now everything should be set in place. Browse to: http://localhost:8080/document-indexing/upload/show to upload a document. Any kind of document will do, as long as it contains some text. When the uploading went alright, the document will be available in the Lucene index. You can test this by going to http://localhost:8080/document-indexing/searchable and type in some text from the document you have just uploaded. The result should be a result page similar to Google.

Friday Repost: Groovy’s @Immutable pitfalls

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Groovy 1.6 introduced some Groovy AST transformations like @Lazy, @Delegate and @Immutable, and there are more. This blog describes some pitfalls of using @Immutable, but to do that, a proper example of @Immutable seems well deserved.

A first introduction

@Immutable, like the name suggests, makes your classes immutable. This means that state changes from the outside, for example by the use of setters, are no longer possible. A good use case for this is when building Commands or DTO’s (also known as Command Query Separation). See for example the following code:

The state of this class should not change. However, in Groovy, this is a bit harder to do than in Java, since Groovy uses a lot of reflection to invoke properties. So the following is quite valid in Groovy:

Being able to do so could cause a lot of unexpected things, and a lot of unwanted behavior! To make sure this doesn’t happen, we can use the @Immutable annotation:

As you can see, two things have changed:

  1. The class is now annotated with @Immutable
  2. The class has been made final, which is a requirement for @Immutable

      When executing the above code again, an exception will be thrown:

      groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: married for class: CreateCustomerCommand

      Seems all pretty and cute, right? However, what would happen when someone with evil intentions (or maybe someone with an old Java habbit), decides to create a setter for this class?

      Now, the code does execute without throwing an exception, and suddenly the code is mutable again. Unfortunately, there’s not much we can do about this, but it breaks the @Immutable contract. There is a small note on this on the @Immutable documentation page:

      “You don’t have to follow Groovy’s normal property conventions, e.g. you can create an explicit private field and then you can write explicit get and set methods. Such an approach, isn’t currently prohibited (to give you some wiggle room to get around these conventions) but any fields created in this way are deemed not to be part of the significant state of the object and aren’t factored into the equals or hashCode methods. Use at your own risk!”

      This text describes the other features of @Immutable: an automatic equals and hashCode method. Since the setter breaks the @Immutable behavior, this is excluded from the equals / hashCode generation. However, this only applies when the fields are private! So be aware of that!

      Conclusion

      My conclusion in this is quite simple: don’t expose state changes to the outside world whenever your intention is to be @Immutable!

      Happy coding!

Friday Repost: Groovy and Null

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

To start, I find it surprising that there’s so little information to be found about this topic. Maybe it’s just me, so let’s start with a test. If you pass all questions, you don’t win much, but then there’s no need to further read this blog I guess.

So, without further ado, question 1.

Question 1

What’s the result of the following code?

Please remember your answer. Now, question number 2.

Question 2

What’s printed in the following code?

Same here: please remember your answer! Next, question number 3. Almost there!

Question 3

What’s the result of the following code?

Same here: please remember your answer! Next, last question!

Question 4

What’s the result of the following code (tip: check question 1!)?

Got every answer written down? Okay, here come the answers!

Answers

Answer 1: “null”
Answer 2: nothing!
Answer 3: A NullPointerException. (java.lang.NullPointerException: Cannot get property ‘name’ on null object)
Answer 4: Another NullPointerException.

All answers correct? Great, you can stop reading now, this blog doesn’t teach you anything. Some errors? Well, maybe there’s a chance to learn something, cause I’ll explain why this happens.

Null object

Groovy has the concept of “Null Objects”. If you do a “null.getClass()”, you will see that the class of “null” is a NullObject, or more precise, org.codehaus.groovy.runtime.NullObject. This class is a used whenever you declare a variable with null, as in “def name = null”. Having a NullObject instead of a Java null reference prevents a lot of NullPointerExceptions for most common operations. However, there are some exceptions!

Answer 1
Because the name is set to null, it’s actually set to NullObject. The toString of NullObject returns “null”, which makes sense, so you’ll see the String “null” printed whenever you call the toString(). It might make sense, but coming from a Java background and not knowing about the NullObject, I was kinda surprised to see it work!

Answer 2
Besides providing some good default implementations for some standard methods, the NullObject also has an iterator() method which provides an iterator over an empty list (Collections.EMPTY_LIST). This allows methods like each, but also collect, find, findAll, inject, every, any, etc (no, this is not a method, it just means etcetera) to safely work on any null collection, which is really nice!

Answer 3
So, while the null object works for most standard methods, there is a limit to the tolerance of it. Any properties or methods not in the NullObject class will still throw an exception when the object is null object. This is explicitly done in the NullObject by throwing an exception in the ‘setProperty’ and ‘invokeMethod’ methods. Just so you know.

Answer 4
Well, already the last anwer in this post. I thought, that when a toString() works, an equals works, a hashCode certainly must also work. And I expected it to return a zero (0). But it doesn’t. It throws an exception as explicitly programmed in the code. To be honest, I didn’t expect this, but I thought it would make sense when using it in a Hashtable, for example. It doesn’t, because a Hashtable already checks if the value is null, I thought this done by the by the ‘is’ method in NullObject (which returns true when comparing the object to “null”), but I have been corrected by Jochen Theodorou: since Hashtable is Java, it has no knowledge about the Groovy ‘is’ method. Also, I haven’t found out why the null object doesn’t return a 0, as I expected, but rather explicitly throws a NullPointer, but at least it’s something to be aware of!

Conclusion

Well, I hope reading this far helped a little in understanding Groovy and nulls. Using Groovy could give you some surprises, especially when you have a Java background, so I hope that after reading this, you’ll have a little less unexpected surprises!

Friday Repost: Remove Groovy MetaClass methods

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Whenever you are using metaclasses in Groovy or Grails unit tests, you might use Groovy’s metaClass power to mock the behavior of methods. However, Grails doesn’t fork the JVM for each test, so a nice side effect of this is that the mocked method will be available for the complete lifetime of the VM. Since Grails first executes unit tests (the test which usually contain the mocked code), and then followed by the integration tests, these tests can fail because you are maybe still using a mocked version of the method instead of the real one! This can often happen when mocking Grails methods, like dynamic finders, or named queries. The trick is to clean up these methods.

This is done by registering the metaClass when extending GrailsUnitTestCase. You’ll have to call the registerMetaClass(clazz) method, which does the following internally:

You can use this code in your tests in the following way, for example in the UserTests.groovy:

This way, the User class is registered with Grails to be cleaned up afterwards. This is done automatically in the teardown of the test, so you’re sure that no custom defined methods are available after running the test.

So, beware that whenever you use Groovy’s meta magic, be also sure to clean it up afterwards, since it might affect other parts of the code. You can check out my other blog post for a good solution to not forget the cleanup part!

Friday Repost: Easy file uploading in Grails

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

If you want to upload files with Grails, there are multiple ways of doing so. However, there’s at least one way which makes it really easy do so, while keeping the code as clean as possible. This blog describes the steps to take to get the file upload to work, while maintaining a very clean codebase.

To get started, first create a new Grails application by running the following command:

grails create-app imageupload

After you have created your application, create a new file called ‘UploadImageCommand.groovy’ in the src/groovy/com/example/command directory

The UploadImageCommand.groovy should look like this:

Using commands for all changes in the system is usually a good thing. This prevents any malicious code from entering the system, and also describes the
intent of the system by using sensible naming for the commands, in this case ‘Upload Image’.

Now we need two more things: the persistent object (the Domain Object) and the Controller. Actually, I would recommend using a Service for all state changes
of the system, because of the transactional support in services and for providing a uniform way of state changes. However, that’s beyond the scope of this blog, so we’ll
leave it for later and focus on the image uploading for now!

The domain class

To create the domain class, typing in the following:

grails create-domain-class com.example.domain.Image

This creates an Image.groovy file in the grails-app/domain folder. Open the file, and type in the following contents:

Also, create a controller:

grails create-controller com.example.controller.Image

This creates an ImageController.groovy file in the grails-app/controllers folder. Open the file, and type in the following contents:

Now, all you need is a simple form which will upload your image (or actually any kind of document). The easiest way to do this is create a new file called
upload.gsp and place it in the web-app directory . Make the file look like this:

Now, type grails run-app

And browse to http://localhost:8080/imageupload/upload.gsp. When you upload a file there, you will see the counter increase in the log file. Clean, fast and easy!