Posts

Learnings of writing Groovy in Action, Second Edition

More than two years ago, I wrote about the effort to revive Groovy in Action. Little did I know that it would take another 2 years of a huge combined effort to get the book out. Maybe not surprising for other authors, and it has been mentioned many times before, but yet; it was an effort. It was quite a time-consuming process to write a book, coordinate with your co-authors, the editors and the publisher, while also keeping in touch with you, the reader, who come up with a lot of greats ideas and publish errata on the forums.

But, having mentioned the effort, it’s such a rewarding effort to finally have finalised the book! If feels like I can finally check one the things on my bucket list, and that is to have my name on the cover of an amazing book. And while I’m very biased, I’m pretty sure this is not only the best on Groovy, it’s also a great book on learning how to program. Topic like functional programming, using collections, testing your software, and more advanced topics like ASTs, creating DSLs, etc are all covered in the book, and provide a great help for beginners in programming as well as advanced software developers.

While writing the book, I’ve learned a lot. Not only about the Groovy programming language, and how to convey our message to our audience, but also about the writing process and the technologies used to write the book. For example, we are using GitHub to store the book, which is a perfect way of keeping track who worked on what, restore versions, etc. However, the book is written in Word. Which is probably the single most horrible way to write a book in. Don’t ever do this! Instead, choose a more open format like AsciiDoc, which is a text document format for writing notes, documentation, articles, books, ebooks, and much more. AsciiDoc can easily generate EPubs and PDFs, while writing in plain text, which is an ideal format for Git. Find a publisher which supports plain text publishing, and preferably AsciiDoc!

Secondly, writing with a (distributed!) team of software developers, who all have limited time due to their family life and their 40 hour workweek, is hard. Coordinating, keeping a focus, it’s hard. When you embark on such an adventure, do a brainstorm session. Create a backlog, keep track of who is working on what, and find a good way of communicating with your team members. Gitter might be such an option, or Google hangout, but in reality, nothing beats being together and having a face to face talk about the progress of your book. It’s not absolutely mandatory, but it helps quite a lots.

Thirdly; keep a focus while keeping the pace. Quality matters a lot, but so does delivering. The first version of Groovy in Action covered Groovy 1.0. The Second Edition originally covered 1.8, but then Groovy 1.9 came out. And then 2.0. So we had to check all the code examples, which, fortunately (and I can really recommend doing so) could be executed automatically. In the end, Groovy 2.4 was covered, which is great for the community, since it being the most recent version of Groovy, but tough on your writing crew. And mostly thanks to the endless energy of Paul King, you can now all enjoy the hard work we’ve put into the 912 pages of Groovy code.

While there are many more things to learn from such an experience, the three lessons above proved to be the most important for me. And while many things could have been done much better, in the end, it’s the results that matter, and difficult roads often lead to beautiful destinations. I think that I speak for the whole Groovy in Action team while I say that we are proud of the result, and we’ve produced a quality product for you to enjoy. I’m proud to have been able to produce such a book, and to have been given the trust of our publisher as well as our audience to deliver the quality you expect us to deliver. I thank you for your patience and your support in delivering this book. We couldn’t have done it without you! Thanks!

Groovy in Action Second Edition PDF and Paper

Only 4 days ago, the paper version (pBook) of Groovy in Action was released, and with the release of the electronic edition (PDF, epub and mobi), all the editions of Groovy in Action, Second Edition, are fully available! So, what’s exactly covered in the book?

The book is divided into three parts. The first part of the book is about the Groovy language. The second part is about working with the Groovy library, while the third part dives into applied Groovy, showing you some best practices around tests and some other frameworks closely related to Groovy.

In the first chapters, the Groovy Basics are covered. The Groovy basics, as the name implies, covers the foundation of Groovy and gives a great start to grasp the language. This chapter is followed by the Groovy Datatypes and Collections, which really show the power and flexibility of Groovy, and showing the start of learning how to use functional programming with the Groovy programming language. This is completed by introducing Closures, one of the core language elements of Groovy. The chapters after this make the first part actually already a complete book, and we could have stopped there if we wanted to limit ourselves to the language only. But the reasons which make Groovy such a compelling choice is not only the flexibility of the language, but also the ecosystem around it.

So, in the second part of the book, the Groovy library, covers technologies like the working with the GDK (the methods added to the JDK by Groovy), Database programming (covering SQL as well as NoSQL databases), XML, JSON, Webservices, and a full chapter packed with tips on how to integrate Groovy into your current software pipeline.

To finish off, the 3rd part of the books (and we’re already 600 pages into the book by now) covers the focus of my work for the last 10 years, which is software testing. A lot of that knowledge has been put into the Groovy book, and I really hope it will prove useful in increasing the quality of your software. Besides testing software, concurrency, always a complex topic, and DSLs, which for me, is one of the most interesting chapters in the book, since it brings a lot the previously covered chapters together and shows some real world examples on how to build your own DSL.

I’d like to thank my co-authors from who I learned a lot about writing Groovy in Action, as well as the Groovy languagehor! We really hope you’ll enjoy reading the book as much we had fun writing it, and if you have feedback, comments or compliments, let us know, we love to hear from you!

Friday Repost: Grails and Bamboo

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In our development environment, we use the complete Atlassian suite. We (or actually I) choose this environment because the products integrates so well together, but also because it integrates good with other products, like SpringSource Tool Suite, or, in our case, IntelliJ IDEA. And finally, the Atlassian suite has excellent support for building Grails projects, which, at the moment, is all we do. Simply put, we choose Atlassian because it rocks!

To integrate even better with the Atlassian Build Server (Bamboo), I’ve created a small script which will update the version number of the application with the Build Number. This way, we always know which build is running where. To do so, I created the following script.

This script will look for a file called ‘build-number.txt’, which is produced by the Build Number Stamper Plugin, and put in the working directory. This file is read by the script, and it updates the version number in application.properties, which will result in a war file produced by the build in the following format ‘..war’, for example ‘todolist.0.1.34.war’. This way you can easily relate each build which bugs you might find, and avoids confusion, for example between testers and developers.

As an extra, we’ve show the current version of the application in the source of our HTML. We do this by reading the version number from the application.properties with the following tag:

This will display the version number of the application, and will allow you to relate issues to a build number, to avoid miscommunication about the deployed application!

Friday Repost: Groovy Test Data Builder Pattern

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Most of us value the quality of our code highly. I do, and I expect you, as a reader, also to care for the quality of your code. Because of that, we write tests. Test which usually contain of 3 parts: the setup, the execution and the verification. And sometimes some cleanup. In this blog, I’d like to focus on the setup part, and try to get rid of some boilerplate code which can easily clutter our tests!

As I mentioned before, we usually write some setup code which serves as input for our test. This code can look like this:

Writing this code can be time consuming, plus the code doesn’t reveal the intention of the test. What are the significant bits in this setup code? Is it the age? Is it my name? Or a combination?

To solve the creation of testdata, there are multiple patterns. On of those patterns is the Test Data Builder pattern. The idea behind the Test Data Builder pattern is that test data should be easily to construct, which is usually accomplished by chaining method calls (also called a Fluent Interface, or a Builder).

I’ve taken the concept to Groovy and made an TestDataBuilder Category. A Groovy Category adds functionality to your code within a limited scope (defined by the ‘use’ keyword). This enables the object which is used in the setup of the test to ‘borrow’ the method of the TestDataBuilder Groovy class, which allows it to construct itself with testdata. This is best demonstrated by the use of an example:

By placing the creation of the data in the scope of the Category, the .build() method becomes available in for the Person class. Calling the build method will instantiate the Person class and populate all the fields with values which all equal true. So numbers will become 1, booleans true, Strings will get the name of the variable as a value, etc.

To use this in tests, and make the code more intention revealing, we can choose to populate some fields with our own values, and have the rest be generated.

The benefits of this is that you save a lot of setup code, and make the code a lot more clear and maintainable. If you want to use this, the code can be found on GitHub.com. (This class was previously called the ObjectMother, an error on my side. The change in name isn’t reflected yet in GitHub). Yes, there are some limitations, like it doesn’t handly all standard datatypes yet (like dates), nor does it populate associations (list with other objects) yet. Have fun, and any feedback on the code would be highly valued!

Friday Repost: Preventing spam with Groovy and Grails

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

When I created the jworks.nl website in Grails, one of my requirements was to have a blog. Since the website is created in Grails, and the Grails application doesn’t integrate easily with a standard blogging solution like WordPress, I decided to create my own blogging solution. But, since this proved to be much more painful than initially anticipated, I migrated the website to a new design and decided to use WordPress instead. However, the plugin creation was fun to do, so I’m sharing my experiences here.

The current blog consists of little more than just a BlogPost class, and a Comment class. This worked pretty well, and I got the blog up in no time. However, my blog was found by the spam bots, and it got spammed a little. In the first couple of days, I got 3 spam messages a day, which could be easily managed by hand (however still quite annoying). Then, the spam load increased to 10, 100 and then a 1000 per day. Then I got really annoyed, and decided it was time for a solution!

My solution was not to choose a Captcha. Captcha’s tend to bother the user, making it harder to post feedback. That’s not something I’d look forward to. So I created my own solution, based on how WordPress does it.

WordPress has their own spam blocking service, which is called Akismet. The nice thing about Akismet is that their service isn’t exclusive for WordPress, but they provide an API so others can also use it! Which is exactly what I did: I created a Grails plugin for Akismet, called (surprise surpise!): Grails Akismet! I’m sure you didn’t see that coming!

It’s very simple to use. Just do a ‘grails install akismet’, add your site plus the Aksimet key to your Grails configuration, and you’re good to go! The plugin provides a Grails Service, which can be injected into most Grails artifacts (like services, domains, and controllers). By using the ‘check for spam’ method in the AkismetService, a call to the online Akismet service is done, which determines if the comment is spam or not. Very easy, and very non-intrusive!

So, if you want to get started with the plugin, please check out the documentation and let me know what you think!

Friday Repost: Indexing documents with Tika and Grails

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In one of the previous blogposts, I described a way to easily upload images by using Grails commands. This blogpost further builds on that, by providing a generic way to index uploaded documents. These documents can later be retrieved by using the search option provided by the Grails Search plugin.

To index documents, we’re going to need two components: one of these components is the Grails Search plugin, as described above. The other component is Tika. Tika is a document indexer library which can index many kinds of documents like Text, Word, Excel, Powerpoint, but also more exotic document formats, like MP3 and FLV. Combined with Lucene, this provides a powerful combo.

So, how do we get this to work? Well, first we need the application from the previous blogpost. I refactored it a little to represent the new generic format better. I renamed all references from image to document (so now we have a CreateDocumentCommand instead of a CreateImageCommand, as well as all other references), and I moved the upload.gsp to the views directory, since redirection and forwarding works better that way. After the refactoring was done, I first installed the searchable plugin by typing:

grails install searchable

I also needed Tika, so I added it to the BuildConfig.groovy in the grails-app/conf directory, like this:

Grails and extra XML libraries really don’t mix well, so to prevent some nasty class loading issues, better exclude them here. What we need to do now is to tell searchable to index our documents when we upload them. This is quite trivial really, we can just annotate the Document class (which in the previous application was called ‘Image’) with a Tika converter. Then we have to create the converter, register it in Spring, and we should be done. So, first things first: let’s annotate the Document class.

In the Document class, add a searchable closure with a converter for the byte array, like this:

We now need to create the TikaConverter. We’ll create a TikaConverter.groovy in the src/groovy/com/example/converter directory. The class should look like this:

This class is responsible for extracting text from any kind of document uploaded to the system. To make this converter available, we have to register it in the Spring context, and then we’re ready to go. To register the Converter in the Spring context, add the following lines to grails-app/conf/spring/resources.groovy:

Now everything should be set in place. Browse to: http://localhost:8080/document-indexing/upload/show to upload a document. Any kind of document will do, as long as it contains some text. When the uploading went alright, the document will be available in the Lucene index. You can test this by going to http://localhost:8080/document-indexing/searchable and type in some text from the document you have just uploaded. The result should be a result page similar to Google.

Friday Repost: Groovy’s @Immutable pitfalls

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Groovy 1.6 introduced some Groovy AST transformations like @Lazy, @Delegate and @Immutable, and there are more. This blog describes some pitfalls of using @Immutable, but to do that, a proper example of @Immutable seems well deserved.

A first introduction

@Immutable, like the name suggests, makes your classes immutable. This means that state changes from the outside, for example by the use of setters, are no longer possible. A good use case for this is when building Commands or DTO’s (also known as Command Query Separation). See for example the following code:

The state of this class should not change. However, in Groovy, this is a bit harder to do than in Java, since Groovy uses a lot of reflection to invoke properties. So the following is quite valid in Groovy:

Being able to do so could cause a lot of unexpected things, and a lot of unwanted behavior! To make sure this doesn’t happen, we can use the @Immutable annotation:

As you can see, two things have changed:

  1. The class is now annotated with @Immutable
  2. The class has been made final, which is a requirement for @Immutable

      When executing the above code again, an exception will be thrown:

      groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: married for class: CreateCustomerCommand

      Seems all pretty and cute, right? However, what would happen when someone with evil intentions (or maybe someone with an old Java habbit), decides to create a setter for this class?

      Now, the code does execute without throwing an exception, and suddenly the code is mutable again. Unfortunately, there’s not much we can do about this, but it breaks the @Immutable contract. There is a small note on this on the @Immutable documentation page:

      “You don’t have to follow Groovy’s normal property conventions, e.g. you can create an explicit private field and then you can write explicit get and set methods. Such an approach, isn’t currently prohibited (to give you some wiggle room to get around these conventions) but any fields created in this way are deemed not to be part of the significant state of the object and aren’t factored into the equals or hashCode methods. Use at your own risk!”

      This text describes the other features of @Immutable: an automatic equals and hashCode method. Since the setter breaks the @Immutable behavior, this is excluded from the equals / hashCode generation. However, this only applies when the fields are private! So be aware of that!

      Conclusion

      My conclusion in this is quite simple: don’t expose state changes to the outside world whenever your intention is to be @Immutable!

      Happy coding!

Friday Repost: Groovy and Null

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

To start, I find it surprising that there’s so little information to be found about this topic. Maybe it’s just me, so let’s start with a test. If you pass all questions, you don’t win much, but then there’s no need to further read this blog I guess.

So, without further ado, question 1.

Question 1

What’s the result of the following code?

Please remember your answer. Now, question number 2.

Question 2

What’s printed in the following code?

Same here: please remember your answer! Next, question number 3. Almost there!

Question 3

What’s the result of the following code?

Same here: please remember your answer! Next, last question!

Question 4

What’s the result of the following code (tip: check question 1!)?

Got every answer written down? Okay, here come the answers!

Answers

Answer 1: “null”
Answer 2: nothing!
Answer 3: A NullPointerException. (java.lang.NullPointerException: Cannot get property ‘name’ on null object)
Answer 4: Another NullPointerException.

All answers correct? Great, you can stop reading now, this blog doesn’t teach you anything. Some errors? Well, maybe there’s a chance to learn something, cause I’ll explain why this happens.

Null object

Groovy has the concept of “Null Objects”. If you do a “null.getClass()”, you will see that the class of “null” is a NullObject, or more precise, org.codehaus.groovy.runtime.NullObject. This class is a used whenever you declare a variable with null, as in “def name = null”. Having a NullObject instead of a Java null reference prevents a lot of NullPointerExceptions for most common operations. However, there are some exceptions!

Answer 1
Because the name is set to null, it’s actually set to NullObject. The toString of NullObject returns “null”, which makes sense, so you’ll see the String “null” printed whenever you call the toString(). It might make sense, but coming from a Java background and not knowing about the NullObject, I was kinda surprised to see it work!

Answer 2
Besides providing some good default implementations for some standard methods, the NullObject also has an iterator() method which provides an iterator over an empty list (Collections.EMPTY_LIST). This allows methods like each, but also collect, find, findAll, inject, every, any, etc (no, this is not a method, it just means etcetera) to safely work on any null collection, which is really nice!

Answer 3
So, while the null object works for most standard methods, there is a limit to the tolerance of it. Any properties or methods not in the NullObject class will still throw an exception when the object is null object. This is explicitly done in the NullObject by throwing an exception in the ‘setProperty’ and ‘invokeMethod’ methods. Just so you know.

Answer 4
Well, already the last anwer in this post. I thought, that when a toString() works, an equals works, a hashCode certainly must also work. And I expected it to return a zero (0). But it doesn’t. It throws an exception as explicitly programmed in the code. To be honest, I didn’t expect this, but I thought it would make sense when using it in a Hashtable, for example. It doesn’t, because a Hashtable already checks if the value is null, I thought this done by the by the ‘is’ method in NullObject (which returns true when comparing the object to “null”), but I have been corrected by Jochen Theodorou: since Hashtable is Java, it has no knowledge about the Groovy ‘is’ method. Also, I haven’t found out why the null object doesn’t return a 0, as I expected, but rather explicitly throws a NullPointer, but at least it’s something to be aware of!

Conclusion

Well, I hope reading this far helped a little in understanding Groovy and nulls. Using Groovy could give you some surprises, especially when you have a Java background, so I hope that after reading this, you’ll have a little less unexpected surprises!

Friday Repost: Remove Groovy MetaClass methods

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Whenever you are using metaclasses in Groovy or Grails unit tests, you might use Groovy’s metaClass power to mock the behavior of methods. However, Grails doesn’t fork the JVM for each test, so a nice side effect of this is that the mocked method will be available for the complete lifetime of the VM. Since Grails first executes unit tests (the test which usually contain the mocked code), and then followed by the integration tests, these tests can fail because you are maybe still using a mocked version of the method instead of the real one! This can often happen when mocking Grails methods, like dynamic finders, or named queries. The trick is to clean up these methods.

This is done by registering the metaClass when extending GrailsUnitTestCase. You’ll have to call the registerMetaClass(clazz) method, which does the following internally:

You can use this code in your tests in the following way, for example in the UserTests.groovy:

This way, the User class is registered with Grails to be cleaned up afterwards. This is done automatically in the teardown of the test, so you’re sure that no custom defined methods are available after running the test.

So, beware that whenever you use Groovy’s meta magic, be also sure to clean it up afterwards, since it might affect other parts of the code. You can check out my other blog post for a good solution to not forget the cleanup part!

How not to add a non-empty item to a list in Groovy

In our code, a colleague (Alard), found the following piece of code in our Grails codebase:

I think the intention of this code is clear (in case of ‘false’ items, add a empty String’), but the outcome was surprising. Take 5 seconds and guess the output of line 4, then read on.

As you might have guessed, the outcome isn’t a list with an empty String (which is displayed as a empty list, but that’s a different story). The outcome of the above is:

[null]

That is because the Groovy << operator has precedence over the ?: operator. In code, it's as if the following had been written:

Which makes it obvious: the item is always added to the list, and the empty String is ignored. Fixing this is easy: just add some clarifying braces: