QCon Sao Paulo 2010

Last weekend I took part in QCon Sao Paulo, the Brazilian edition of one of the main worldwide events for software developers and architects. The conference had a strong focus on issues related to scalability, cloud computing and replication in addition to Java, .NET, Ruby/Rails and Agile. Redis and memcached were two of the most mentioned tools used to deal with scalability issues.

Nick Kallen, Systems Engineer of Twitter, talked about how he designed and evolved solutions to deal with some of the fundamental types of data of Twitter: tweets, timelines and social graphs. Each of these real time data had different datastore needs. He discussed how the original solution to store the data in MySQL did not scale well and then went on to present the datastore strategies they created to meet their specific needs.  They exploited techniques like master-slave replication with memcached for reads, data partitioning, temporal locality and indices. A recurring pattern to achieve efficiency and scalability that was identified was to partition, replicate and index the data.

Guilherme Silveira talked about REST, semantics and the future of the web. He presented some ideas that can be used such as exploiting the power of links, microformats with hypermedia, addition of semantics and the existence of a shared vocabulary.

Randy Shoup shared some of the best practices for large-scale web sites based on his experience as a chief engineer at eBay. Some pieces of advice were: 1) partition everything; 2) asynch everywhere; 3) automate everything; 4) remember that everything fails. He’d already presented this talk at QCon San Francisco in 2008 and it can be watched here.

Douglas Crockford presented about the future of JavaScript (or ECMAScript as he pointed out). He went through some changes that are being proposed to the language, especially better math support.

Rodrigo Yoshima talked about the method war we are witnessing these days: Scrum vs XP; Scrum vs Kanban; PMBok vs Scrum; Lean vs XP, etc. He reminded that the most important value of Agile is to deliver value more rapidly. We can be applying a handful of agile best practices but if we don’t stick to that one principle we won’t truly be Agile. He also pointed out that there is no such thing as a closed scope in software development.

Paulo Silveira talked about the blurry distinction between architecture and design; and also between design and implementation. He quoted several renowned authors and his conclusion was that “architecture is every decision that impacts in a big trade-off and can or not be difficult to change”. It also can be evolutionary (last responsible moment). When questioned about whether it makes sense to be a “java architect”, he replied with a quote by Joel Spolsky that says we need at least one architect with great experience on the chosen platform. Paulo then went on to discuss (with some real examples) the role of  design as a key element to being able to make changes both to the implementation and the architecture of a software system.

The event had several other good talks that I won’t describe here. The organizers are working to make the talks available on InfoQ.

Watch out for TODO comments!

Say you have a class with 40-something methods, doing a number of different things… You, as a great software developer, identify that as a bloater smell and wisely conclude that code cries out for a refactoring. Sounds pretty reasonable. Nevertheless, you’re neck deep in work now trying to meet the deadline for the next release. So what you do? You simply leave a TODO comment next to the code for you to perform that change as soon as you have enough time available.

 // TODO: This class is doing too many things. Refactor!
 public class BloatedClass {

    // 40-something methods

}

That’s it! Now you move on with your next task feeling good about yourself because you think you’ve already handled the problem “appropriately”.

“What’s wrong with that?” – you may ask – “after all, I do intend to deal with that later” . Come on, my friend! Let’s be realistic and face the facts! Chances are you’ll forget about that comment and it will remain there forever since neither you nor anyone else on the team will have the guts to carry out that refactoring afterward for fear that something might stop working. This happens because if you remember that comment, you’ll probably already have lost the general context that surrounds that code and thus you may not have enough confidence to refactor. Worse comes to worst when there isn’t a good suite of automated tests to back you up. As a consequence, your fellow workers will have to work and get along with that poor legacy code.

Bottom line: it’s best to try to keep yourself from adding TODO comments to your source code in the first place (in fact, this applies to code comments in general); but, if you do, make sure you don’t forget about them (Eclipse has a “Tasks view”, which can be useful to track those notes). Those comments might be indications of technical debt that you need to be mindful of, yet more often they’ll just be deodorant for poorly written code.