New year’s goals

The beginning of a new year is a favorable time to look back and reflect on the events that happened in the past year and take stock of our life. Many people start their year with a lot of excitement and full of new goals and resolutions. But as time goes by, they get overwhelmed by the business of daily life and little by little start compromising on what they originally had set out to do. Worse, there are people who never do any planning whatsoever! They just let it be. There is a quote that says “if one does not do any planning, one is planning to fail.” The Law of Entropy is unforgiving, my friends!

So, clearly it is healthy and important to be constantly establishing new goals for every area of our life and commiting to them. So, how can one come up with reasonable goals and not lose track of them? Let me give you some suggestions.

1) Make sure your goals are S.M.A.R.T, i.e., specific, measurable, achievable, realistic and timely. Pay special attention to the achievable and timely factors. For example, instead of: “I want to read more this year”, put something like:  “I want to read three books this year: book ‘abc’, ‘def’ and ‘ghi’ “.

2) For each goal, design a specific plan to achieve it. Goals (and dreams also) without a plan to achieve them are just desires, just wishful thinking. For each goal, write the steps necessary to reach that goal. For example, “on Mondays and Wednesdays I’m gonna read for two hours the book ‘abc’ “.

3) Make it visible, inspect and adapt! Put your goals and plans in a place where you can always see and review them. Assess how much progress you are making and make the necessary adaptations in your plans. This inspect-and-adapt cycle is very important to keep your goals feasible (and, consequently, keep you motivated). These ideas were taken from the Scrum framework. Therefore, if something is not working, change it! Remember that “it’s foolishness to do the same things over and over again expecting to obtain different results“.

Examples of areas you might consider when drawing up your plans: spiritual, physical, familiar, professional, personal growth, social, leisure and financial.

I’m already outlining my goals and plans for this next year and this will be part of my strategy. Have you done your planning? If not, what are you waiting for?

“To man belong the plans of the heart,
but from the LORD comes the reply of the tongue.”

(Proverbs 16:1 – Bible)

HAPPY NEW YEAR!

———–
Picture from: http://www.city-data.com/forum/photography/397255-interesting-project-6.html

Book review: Managing Humans

I’ve finished reading Michael Lopp’s book “Managing Humans – Biting and humorous tales of a software engineering manager.”

The book is full of anecdotes describing the world of a software engineering manager. Despite the rather slangy style of writing (sometimes using bad language), the book encompasses several facets of management and is full of thoughts useful both for managers and for those who deal with managers (that is, all of us!).

It is divided into three parts: 1) The Management Quiver, 2) The Process is the Product and 3) Versions of You.

Next I transcribe some of my favorite quotes from the book along with my comments.

• “Managers who don’t have a plan to regularly talk to everyone on their team are deluded”. Everyone likes their opinions to be heard and considered. When managers don’t talk and listen carefully to what their employees have to say, besides disheartening the team, the company is bound to lose great talents and ideas.

• “The organization’s view of your manager is their view of you”. Therefore, it’s quite important for one to work out the relationship with their manager. Where does your manager want to get? What does he want to achieve? What is important for him? As an employee, you need to figure that all out and make sure that your manager is making progress, because if he succeeds you eventually will succeed. (Conversely, if a manager wants to be successful, he needs to consider these questions in regard to the members of his team.)

• The author talks a lot about meetings and the different kinds of people that take part in those meetings. In meetings, there are two major sorts of characters that need to be identified: players and pawns. The former is directly interested in the result of the meeting, while the latter typically gives no contribution. Knowing how to identify and deal with these roles is important in order to make the most of a meeting.

• In the last part of the book, the author described the several meeting creatures (another spectrum of sorts of people found in a meeting). Especially important is the synthesizer, which is the person who can gather all the sparse information thrown by people and turn it into clear-cut sentences understood by all. In my company, during the planning meetings of our sprints (we’re running Scrum), we usually refine the backlog along with all the team, scrutinizing each user story. In this process, several people make comments and we can spend quite some time discussing a single story. When the time to give the estimate comes, it’s extremely useful to have one person be the synthesizer, summing up what the story actually consists of.

• Information conduit – “for each piece of information you see, you must correctly determine who on your team needs that piece of information to do their job”.  A manager selfish when it comes to information simply won’t be completely trusted by the team. Worse, as the author points out, “in the absence of information, people will create their own.” I also considered quite important the advice to give each piece of information we’re passing on a bit of our personal context. For example, how many times have you sent out an e-mail with a link to something interesting you read, but without adding anything to it, just the raw link? I’ve done it many times. Now I try to at least summarize what I am forwarding and preferably give some personal opinion.

Overall, I would say that the idea of using tales and humor to illustrate the ins and outs of management was a good one, but that could have been better explored. Nevertheless, the book has more illustrations and good advice for anyone interested in this topic (there’s also a humorous glossary of management at the end whose explanations are quite direct and helpful).

Test Automation for the Persistence Layer with FIT, DBUnit and HSQLDB – MundoJava 38

The edition 38 of MundoJava magazine is out! In this issue I wrote an article about Test Automation for the persistence layer using FIT, DBUnit and HSQLDB.

The article is the result of an experience on a past project I worked on recently. It was a simple desktop Java system whose goal was to read data from a bunch of database tables and configuration files and generate several output files formatted in a specific way. We had a simple persistence layer with DAOs implemented with JPA/Hibernate.

The entire project was developed with automated tests (unfortunately, most of it did not use TDD, ). In order to test the DAOs (some of which had rather complex SQL queries), we had integration tests written with JUnit and DBUnit along with an in-memory HSQLDB database. For simple DAOs (and respective domain objects), that approach worked quite well. However, for the more complicated ones, we started to have a few hard-to-understand tests, since a lot of fixture code was being required to set up the data needed for the tests.

So, we wondered if there would be a way to express the integration tests for our DAOs in a more readable way. Now, our project dealed with accountancy data, so we thought that it would be perfect if we could express some of our tests in tables. That was when FIT came to our mind. FIT is a tool which allows one to write automated tests in a tabular format (it uses HTML tables). Just what we wanted!

So, we were able to successfully convert our automated JUnit tests of DAOs into highly readable HTML fixture tables. Now let’s talk in more details about how our previous strategy was and how it changed with the introduction of FIT.

Using DBUnit, our HSQLDB test database was set up with the test data in an XML dataset file before the execution of each integration test. Then, we would have to construct the list of objects that would be the expected result of the call to the DAO’s method under test. Besides being difficult to read (though we experienced some improvement by using test builders), those objects evidently had a lot of duplication with the data in the XML dataset.

Our approach with FIT was to put all the necessary setup data together with the test itself in the HTML document (each DAO has its corresponding tests in an HTML), removing the XML datasets. The first section of the document, which is the configuration session, starts with the tables containing the data to be inserted in the test database. The last section of the document contains the test scenarios with the actual tests and expected results. The HTML test document was implemented using the Flow Mode of fitlibrary’s DoFixture. Inside the Flow Mode, for the first section of the document we used the SetUpFixture (also from fitlibrary). The corresponding Java glue code for the SetUpFixture, in turn, used DBUnit’s DefaultDataSet to programatically build the dataset with the contents of the HTML setup tables (which then were inserted in the HSQLDB database just like before). We ended up creating a few utility classes to enable DBUnit to configure the test database with the data from FIT tables.

Even though FIT has been designed for the automation of acceptance tests (hopefully with customer colaboration), we found it pretty effective to be used to test data access code of data-driven applications. At the end, we had highly expressive integration tests that looked like executable specifications.

In the article, our approach is fully described using a working example and showing all the related code. Comments and suggestions are always welcome!

Caelum Day in Rio

Last Saturday I participated in Caelum Day in Rio, an event about Software Development that had great presentations and several speakers.
Phillip Calçado gave the keynote “All I wish I knew before I had become a tech leader” (my own translation of the title in Portuguese). He actually just reinforced the importance that in software development only one thing guarantees survival: delivering value (all the time, at the exact moment, before it is too late). He mentioned that one of the most important tasks of a tech leader is to avoid the unexpected. To do that, it’s necessary to build barriers, whose objective is to encourage the feedback cycle. Barriers come in five layers:1. Development, 2. Integration, 3. Verification, 4. Validation and 5. Production. The barriers on each of these layers can be achieved through the use of established best practices, to name a few: TDD, fast builds, continuous integration, close relationship with the customer, kickoff-play-walkthough model, DDD, incremental and frequent delivery, and simulation environments.  Phillip also emphasized that barriers will be broken, but what we want is to know when that happens.

Fabio Kung talked about Cloud Computing, a topic that is in fashion in our industry recently. He went over the several aspects of cloud computing: Infrastructure as a Service – IaaS (eg.: machines, hardware, network, etc.) , Platform as a Service – PaaS (Google App Engine, Amazon EC2, etc. ), Software as a Service – SaaS (GMail, Google Docs, etc.). The great majority of Cloud Computing relies on virtualization. With virtualization, we can solve common problems related to waste, provisioning and costs. With Cloud Computing, we can go a little beyond and also tackle the problems of capacity planning, maintenance and availability. What I found interesting is that only 1% of the worldwide major applications are running in the cloud, mainly because they need to have control over their own infrastructure. The great niche of cloud computing are small and medium-sized applications, where one doesn’t want to care about infrastructure and platform issues. It sure was a very enlightening talk, full of interesting stuff. BTW, Kung also showed some funny videos of GOGRID, which can be seen here.

I also watched some cool short presentations: Paulo Silveira, on Java Persistence API; and Rafael Martinelli on Adobe Flex. Sergio Junior e Luiz Costa talked about RESTful webservices in Java and Caue Guerra talked about how the adoption of  Ruby on Rails in Brazil has increased tremendously in the last years (especially in 2009). Guilherme Silveira talked about nice features of the Java web-framework VRaptor 3.

To finish the event, Nico Steppat delivered a great presentation about NoSQL and non-relational databases (examples of those include: SimpleDB, CouchDB, MongoDB and BigTable). Nico talked about how difficult it is to scale out relational databases and how non-relational databases can effectively address this problem. He quoted the Brewer’s CAP Theorem, which says that we can have at most two of the following properties for a shared-data system: Consistency, Availability and Partition Tolerance. Relational databases are good to achieve consistency and availability, whereas non-relational databases are better suitable to achieve the other two combinations.

Rails Summit Latin America: summary of the second day

The second day of Rails Summit Latin America 2009 was awesome!

Richard Kilmer talked about HotCocoa and MacRuby for the OS X. I found it interesting to see the efforts of Apple in order to make Ruby so great a language for their OS as Objective-C. I also got to know a little better about the Mac world, which is quite unknown to me.

Carlos Vilela gave an  overview of the use of  Ruby on projects at ThoughtWorks. It was basically a summary of the research Martin Fowler did concerning Ruby projects since 2006 at the company. What is remarkable about this research is that the majority of projects were considered to have made the right choice as to the use of Ruby and many developers reported a productivity boost of at least twice. Nevertheless, there must be some care regarding the overuse of metaprogramming.

Marcos Tapajós gave a presentation on Apache CouchDB, a document oriented database that can be accessed via a  RESTful JSON API. I had already heard about it, but it was quite new stuff for me. He also commented on the tools that allows for easy integration with Rails. From what I understood, it´s a great option to achive highly scalable systems. I definitely want to experiment with it in the near future.

Leonardo Borges, now a consultant of ThoughtWorks, talked about how to introduce JRuby in a Java project. He actually showed an approach to evolve an existing Java application using Rails. The idea was to package the application in a jar file and put it in the loadpath of the Rails application.

Vinícius Teles spoke about his path until the establishment of his product be on the net. Actually, he presented a short bio about his life since college and shared useful pieces of advice and lessons he learned in his professional (and personal) life. I strongly recommend watching his talk, which is available here (in portuguese).

To top off, Obie Fernandez delivered an inspiring talk on the art of application development. He endorses application development is a craft and in order for one to be excellent there is no other way but practice, practice and practice.

Overall, I really enjoyed attending this conference and look forward to the next edition!

Rails Summit Latin America: summary of the first day

The first day of Rails Summit Latin America was just great. In this post, I’ll try to give an overview of  some of the talks I watched.

Chad Fowler opened the conference with the keynote “Ruby on Rails Insurgencies”. He pointed out that about 50% of our lifetime is spent on work, so we’d better do it the right way (and a lot of people are not doing the right way).

He started off commenting on how he and a group of folks were in the bleeding edge when they began the first RubyConf, back in 2001. Out of this small number of pioneers, five were authors of the Agile Manifesto and eight were well-known book authors. The point is that there were really good people investing their time on “serious stuff”.

He talked about the reasons why companies are resistant to new technologies and ways of doing things (value rigidity). He suggested the reading of “Zen and the Art of Motorcycle Maintenance“. The companies are “guarded by the trolls”, who have FUD :
- does Rails scale?
- ruby is slow!
- we can’t find developers who know
- ruby is unsupportable
- rails is just MVC
- it’s insecure
-…

Chad clarified those misconceptions and then started giving some advice on how to push Ruby/Rails technologies in a corporation. The secret is: do it gradually, so you don’t shock the stakeholders.

Following Chad, Gregg Pollack gave the keynote “On the edge of Rails Performance”. He went over several Rails tools to tackle performance issues of Rails applications. Among them were: bullet, Rails indexes, Scrooge, Rack bug and rubber capistrano.

Carlos Brando spoke about how he created a framework for the development of opensocial applications using some Rails plugings. He discussed decisions regarding architecture (push-based vs. pull based), database ORM strategy, code generation and templates. What I liked about the presentation was the approach he gave on how easy it can be to create a framework in Ruby, reusing existing libraries (after all, Ruby is not all about Rails at all).

In the afternoon, Glenn Vanderburg spoke about his tool for Fuzz Testing of Rails applications: Tarantula. I didn’t know this type of testing until this talk, so it was sort of new stuff for me. Tarantula simply crawls a Rails app, traversing links and looking for potential problems (broken links, XSS and SQL Injection vulnerabilities, etc.). I’m gonna check if there are similar tools for Java applications also.

Following the subject of testing, I saw David Chelimsky talk about some cool features of RSpec. That was a great presentation on nice features of the testing tool like: pending capabilities for tests, implicit subject, explicit subject, custom matchers and wrapped assertions. He also mentioned Spork/Autotest, which are tools that quickly execute all RSpec tests of an application every time a test file is modified (and saved). He is the author of the RSpec Book, which is about to be released.

Fabio Akita gave a rather unusual lecture whereby he analysed concepts from the chaos theory and how they relate to agile and organizations in general. He wrapped up by saying that success does not come from a plan, but from the adaptability to the environment (now this has a whole context and is not an excuse not to plan, otherwise we’ll be just blowing away bythe wind).

The last talk was by Matt Aimonetti, who spoke about the work being done on Rails 3. He discussed several aspects about Rails 3, also showing it in the context of the various existing implementations of Ruby (MRI, IronRuby, JRuby, MacRuby and Rubinius).

To finish the first day, we had a pleasant session of lightning talks. The presentations included: 1) Guru-SP (São Paulo’s Ruby User’s Group), Ruby Inside Brasil,2)  how not to make an event (very funny talk of the people of the Oxente Rails conference), 3) Aldinho (a 11-year-old boy) made his 15-minute-blog using Rails, devise: flexible authentication solution for Rails apps, 4) Guilherme Chapiewski and Henrique Bastos talked about how small acts make great revolutions (it was about how the developer’s community of Rio de janeiro has grown through some simple iniatives like Coding Dojo Rio, PythonCampus, hackframps, Dev in Rio and others), 5)  a real time web application for a basketball marathon. There were a few other speakers also and Daniel, who closed the session playing two funny songs about automated tests and Fabio Akita.

Rails Summit Latin America 2009

Another great event which I´m going to take part in the next few days is the Rails Summit Latin America, also held in São Paulo. The first edition of this conference received a lot of praise from the attendants, and this second edition is expected to be even better.

There will be sessions covering Ruby and Rails technologies as well as Software Development and Agile in general. The list of speakers include Chad Fowler, David Chelimsky, Glenn Vanderburg and some other great developers.

Later I´ll try to post comments about both conferences.

Encontro Ágil

This weeked I´m going to São Paulo to take part in a great event about Agile Methods, Encontro Ágil (Agile Meeting). The event will have many sessions on a variety of subjects. On Saturday, the whole day will be filled with 4 simultaneous sessions! It will be difficult to choose which one to attend, since there are lots of interesting topics. On Sunday, the special guests Jutta Eckstein and Joe Yoder will minister keynotes and tutorials.

It will certainly be a great opportunity to discuss Agile and exchange experiences with great professionals. Hope to see you there!

RioJUG: The myth of agile teams

Yesterday I attended the monthly meeting of the Rio Java User Group, here in Rio de Janeiro – Brazil. The speaker was Victor Hugo Oliveira, the IT Director of Concrete Solutions.  In his lecture “The myth of agile teams”, he talked about the enormous difficulties in working in agile teams and how to deal with the common misconceptions in order to achieve better results. What he talked about applies to any agile method and he mostly used Scrum as an example. In this post, I´m going to sum up what I learned from his talk.

He started off by quoting Ken Schwaber:

Scrum’s productivity stems from doing the right things first and doing those things very effectively

He talked about the myth of hyper-productivity that usually surrounds agile methods. Agile is being successful not because of hyper-productivity, but mainly because of the continuous and early delivery of ROI. It´s true that people usually get more motivated to work with more lightweight and effective methodogies than more heavy and rigid processes. This boost of motivation often implies an accompanying increase in productivity. But that´s not the main goal in the first place, so avoid unrealistic promises of productivity when it comes to agile methods.

He then goes on to speak about some of the myths and issues regarding teamwork:

• harmony vs. creation: harmony = success ? Not necessarily, creative teams need some conflicts.
• more people = more productivity: the project needs to have a manageable number of connections (points of communication between people), this is a quadratic function (he also mentioned the Ringelmann Effect)  Usually, it´s good trying to follow the 1-digit rule, that is, teams with up to 9 people (if the project demands larger teams, it´s necessary to break down the project into smaller cohesive subprojects). Worse comes to worst, as states Brook´s law, “adding manpower to a late software project makes it later”.
• stagnate vs. stabilize: people don´t stop evolving because they are in the same team for a long time. Actually, working in teams for a long time can significantly contribute to increase productivity because of the high team integration. Stagnation has to do with other things, like lack of challenges, for example.
• unpredictability of emergent behavior: a group of people is not the same as a team. Teams need to be looked after. Having a bunch of people with no coordination, expectations and limits is a recipe for disaster. More importantly, a true team needs to have a common goal, a glue that sticks people together. Examples of common goals may include things like project, ROI, risk, the product backlog (Scrum), and so forth. A team can only be defined as self-managed as long as it works with a common goal.
• project needs: what is necessary for a project to be successful? team capacitation! this is quite obvious, but always important to be recalled: the team members need to have the necessary skills to work in the project (ex.: architecture, development, business knowledge, tests, GUIs, etc.). That´s a reason why one of the pillars of Scrum is multi-disciplinary teams.
• size of the challenge:  no challenge leads to complacency, too big challenges lead to discouragement. Choose the right-sized challenge for your team!
• work structure and institutional support: that means a good environment, transparency in the rules, well-defined teams and tasks. Companies should give support so that people can work well together (not necessarily without any conflicts). Teams are a unit both in success and in failure.

The speaker concluded his talk by saying that a project shoud have:

• clients with well-ajusted expectations;
• managers with team support;
• prepared and available coaches;
• a conscious team;
• an aligned institution.

He again mentioned Ken Schwaber by saying that Scrum works primarily at the level of the team and Scrum can also be considered a kind of social engineering, which fosters cooperation.

Testability: How much logic belongs in the constructor?

The other day my colleague and I were pair-programming and we came across a dilemma regarding the design of a class which was responsible to load the configuration properties of our application. The class looked like the following (a little oversimplified):

class ApplicationConfig {

private Connection connection;

public ApplicationConfig(String connectionUrl, String driver) {
try {
Class.forName(driver);
this.connection = DriverManager.getConnection(connectionUrl);
} catch (ClassNotFoundException e) {
throw new IllegalArgumentException(“Problems while loading JDBC Driver.”, e);
} catch (SQLException e) {
throw new IllegalArgumentException(“Problems while creating JDBC Connection.”, e);
}
}

public Connection getConnection() {
return connection;
}
}

In order to achieve better testability by minimizing side-effects, it´s highly recommended to strive for simple constructors that do nothing other than making simple assignments to fields. As Misko Hevery comments on his Guide for Writing Testable Code, constructors should do no real work. Mark Needham, in turn, advocates the simplification of constructors by delaying calculations. He points out the idea of reducing field calculation in the constructor, only calculating those values when they´re actually needed, which generally is a while after the object has been constructed.

Following this advice, we moved the creation of the connection to the getConnection() method, which led us to the following:

class ApplicationConfig {

private String connectionUrl;

private String driver;

public ApplicationConfig(String connectionUrl, String driver) {
this.connectionUrl = connectionUrl;
this.driver = driver;
}

public Connection getConnection() {
try {
Class.forName(driver);
return DriverManager.getConnection(connectionUrl);
} catch (ClassNotFoundException e) {
throw new ConfigException(“Problems while loading JDBC Driver.”, e);
} catch (SQLException e) {
throw new ConfigException(“Problems while creating JDBC Connection.”, e);
}
}
}

OK, now we have a simple constructor. However, the problem with this design is that we will only be able to check if the connection arguments are truly valid in the getConnection() method, which is too later after the object is constructed (you can also notice the different exceptions that are used). The previous design allowed us to signal the problem as soon as possible (fail-fast), which helps to improve bug tracking.

So the issue is whether we should favor the fail-fast rule or the testability. We opted for the latter since this configuration code is pretty much under our control, that is, the risk of passing invalid arguments to the constructor is low.

Just as an aside: we could eliminate the problem altogether by injecting the connection into the ApplicationConfig class, but the point of having the ApplicationConfig class in the first place was that it could be easily reused in other places, removing the duplication of the boilerplate code. The purpose of the class is to be a utility class to load configurations, so we found it reasonable for it to have the responsibility to instantiate a connection.

I´d appreciate comments and suggestions on how to address this issue.