Archive for the ‘Technology’ Category

Behind the Scene: Development Environment

Friday, July 13th, 2007

As discussed in our Application Platform and Server(s) articles, our runtime is based on Java, MySQL and Linux; but what, you may ask, are we using to build this runtime?

Here is a quick peek at our development environment:

IDE: Eclipse 3.3 with AJDT 1.5, Mylyn, and soon Aptana.

AJDT 1.5 is an Eclipse extension to support AspectJ programming. It is a very well done eclipse extension based on the AspectJ library, which makes Aspect programming almost as if it were part of the Java language. Thankfully, the bug we had encountered in Eclipse 3.2/AJDT 1.4 has been solved.

We have not started using Mylyn yet, but are planning to use it since we are using Trac as our issue-tracking system.

We are also planning to use Aptana as soon as it officially supports Eclipse 3.3, although the default Eclipse 3.3 HTML/CSS is pretty good.

Source Control: Subversion, TortoiseSVN

Subversion is probably a no-brainer as a free source control. Subversion http support (and especially Apache integration) makes secure remote access configuration a really easy step.

We are big fans of the TortoiseSVN Windows extension, which makes Subversion accessible to almost any computer user. This is one of the reasons that we decided to use Subversion as our content-management back-office tool (article about our back-office tools coming soon).

Bug Tracking, Wiki: Trac.

Life before Trac was complex and fragmented. You had to look for a Bug System, a task management system, a Wiki, and eventually a Source Control web front-end. Trac took the approach of offering a simple integrated approach for bug tracking, collaborative editing (i.e. Wiki), and web access to source control. Trac is also easily extensible, which is allowing us to tailor it exactly to our needs. A simple customization can sometimes make a big difference!

Anyway, big thanks to the Trac team for their approach and execution. Looking forward to the next version.

Build System: Ant and CruiseControl

Like any Java developer, we are using Ant to build our application. Its integration with Eclipse and Tomcat is just what we need. We are also planning to use CruiseControl as a release management tool.

Testing Framework: JUnit 4

We are using JUnit 4 for our unit test. The Fixtures (i.e. @Before/@After) features have allowed us to build a simple and efficient test framework for our application. In short, we are using @BeforeClass to create the application context, and @After to clean the database.

Advice needed: While we are comfortable with our JUnit choice for unit testing, we are not sure about what we should use for our HTML/HTTP testing. The httpUnit project seems to have been put on hold. Any advice would be appreciated.

User Interface Design: Adobe Photoshop CS3, Flash CS3

If you can afford them, these are just the best tools to build your user interface assets and screen mockups. We are using Photoshop CS3 to design the main screens and graphic assets, and Flash CS3 to build screen flows for early User Interface testing. We might use Flash/Flex Builder in the future to build some application components.

Disclaimer: I once worked at Adobe as Director of Product Management for Flex and ColdFusion. So, I might be a little bit bias. Nevertheless, Photoshop has always been among my top three favorite desktop applications (along with Excel and Eclipse).

Our Server(s)

Friday, June 22nd, 2007

Well, we have one–just one, actually, but it’s a pretty good one.

Here is the configuration:

  • 1 Unit (Dual processors capable)
  • Intel Quad core E5345 (Single processor)
  • 16GB RAM (DDR2-667 Fully Buffered RAM)
  • 1.5 TB SATA in Raid 5 (corresponding to 3 x 750GB).
  • We have an auxiliary 160GB disk as floating drive.

Price: Between $5k and $7K depending on how you build it.

We evaluated big brand names and smaller brands, such as Penguin Computing. After careful evaluation, we decided to go with Penguin Computing. We are definitely happy with our choice. Penguin Computing has great experience with Linux, and their units are extremely well-designed and truly open and extensible.

Here is our “serv01.sportner.com” server (when it was in my garage):

server01

As far as OS, we are have decided to take a risk and go with Microsoft Windows Server 2008 Beta 3 and SQL Server 2008 CTP… no, just kidding…

We are using CentOS 5, which is an amazing OS (thanks to RedHat and the CentOS org), with Xen. Today, we just have a Dom0 and one DomU for the blog and the service. The first additional domU will probably be dedicated to mail serving (mail for the service itself; we are using Google Apps for our internal email system).

Tips: Always double-check the price of your complete server. Do not look at the discount numbers. Big brands will always start very low and climb very, very fast. Also, some brands tend to have little “locks” in their servers. For example, some servers have proprietary drive bays which force you to buy all your drives from them. We also have a smaller development and back-office server, with a similar software configuration, but much smaller RAM and Disk. Penguin Servers are truly open, and come with all the bays regardless of the number of disks you originally purchase. If you want more information about Penguin Servers, feel free to contact Tom Tran.

The Application Platform

Monday, June 11th, 2007

On the geeky side, here is some information about application platform choices we made. First, we have decided to write all of our server code in Java. While other development platforms sound attractive, we think that Java has come a long way from its heavy and complex days, and now offers a very complete and mature set of tools and libraries for high scale consumer applications. Find below a snapshot of our current application architecture:

Sportner Applicatoin Architecture

From a bottom to top:

  1. MySOL 5 and FileStore are used for data and file storage. At this point we are not clustering the database, and have a simple “File Segmentation” based on user ID’s. Long term, we will have to find a way to “split or cluster” the database.
  2. Java 1.6: We were pretty satisfied with Java 1.5, but Java 1.6 seems to perform better, and includes Jaxb2.0. It’s also good to start your development/production cycle with the latest version.
  3. Tomcat 6.x: We picked Tomcat 6.x mainly for the reason above (i.e., to start with the latest version). We do not really use any deep Servlet or JSP features at this point. So far, we have been very happy with Freemarker, and our first JSP 2.1 evaluation showed that it still did not meet some of our requirements (e.g., template in the database).
  4. Hibernate 3.2: Who would not use Hibernate 3.2 when developing a database-driven application? Actually, there might be some reasons to not use it, but for now, we have determined that the cost/benefits analysis favors using it. We know we will have to optimize the SQL Statement some day. However, we also think that if we want to scale, we will have to cache, which will make the O/R performance issues not such of a bottle neck.
  5. Lucene 2.x: Who would not use Lucene to provide search capabilities? There might be some reasons, but we do not know about them. We might use it for our cache strategy as well (not sure yet).
  6. Jaxb2.0: This is actually part of Java1.6 now, but from an architecture standpoint, it belongs to this level. While Jaxb1.0 was a typical “EJB2.1 over-designed” library, Jaxb2.0 takes advantage of annotation to simplify a lot the marshalling and unmarshalling rules. We have not done heavy performance testing yet, but the new APIs work like a charm (no need to specify Schemas anymore!)
  7. AspectJ 1.5: We are big fans of Aspect programming. However, given that it is not part of the Java language, we are using this library very cautiously, mostly to propagate Interface implementation such as CreationStamp and UpdateStamp. We are using AJDT for Eclipse, which works very well, except for a little bug which is very frustrating. Sometimes we have to clean the project to be able to compile, so it does get frustrating after a while… looking forward to Eclipse 3.3.
  8. Freemarker: Freemarker is a modern template engine a little bit like Velocity but more “modular”. It is extremely extensible, and flexible, and while JSP2.1 has some nice features, Freemarker is still the most flexible and mature Java template engine that we know of.
  9. Yahoo! UI: For business and usability reasons, our application will follow the common “web page model”, but it will also offer some subtle and advanced AJAX interactions. Yahoo! UI is a solid and complete AJAX framework to incrementally add dynamic behavior to your web page. At this point, we are not sure how many of the standard Yahoo! UI widgets we will be using, but we will definitely use their core libraries. At some point, we might make some more advanced components, and Adobe Flex will be a natural fit for those.

Sportner’s architecture has two main layers:

  1. Universal Service Platform (AKA USP): This is not a “Universe Universal” but rather a “Sportner Universal” platform. Basically, this layer contains all of the generic parts of the system that do not have any specific connection with the given “Sportner” application. This allow us to have a clean abstraction, and to have a platform upon which we can build auxiliary applications if necessary (we started to write a “Name Finder Application” during our days of desperation).
  2. The Sportner Application (AKA SPR): This is the main Sportner application on top of the USP layer. It has all of the templates and logic specific to the application.

This architecture is pretty basic. The trick is to find the right compromise between using existing libraries and building the right infrastructure. One thing that people could ask us is why we are not using Spring? The main reason is that we do not need it yet. Our first quick look at some Spring components showed us that they might be overdesigned for our needs (at least for now). Our “USP” Layer is very thin, and we are working to make it thinner. At this point, we do not really need more. Time will tell, we are open minded (at least, we try to be).