Sunday, September 19, 2010

Java IoC containers and classpath scanning (or what I’ve been looking for from .NET for months)

Frustrated with the typical way I saw IoC used in Java where every example I found involved thousands of lines of XML
and/or Java code to configure Java beans or components.  This is very different than IoC typically used in .NET where most IoC containers allow
you to "autowire" in their terminology up every class in an assembly with a couple of lines of code.  Having been coding in that fashion
for several years in .NET I was dismayed when none of my fellow Java coders that I worked with or knew personally had any concept
of the equivalent functionality, and instead informed me the IDE would be my help in maintaining these massive XML files.

Not content with their answers I burrowed into Spring, Guice and PicoContainer docs and found "classpath scanning" which is roughly
equivalent to "autowire" in .NET. Below is an example of this in Spring 3:

MyStuff.java (my main class note his has 2 dependencies which you can view on my github repo)


package org.foo;



import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.stereotype.Service;



@Service //one way of marking this as a component to register

public class MyStuff {

    private final DependencyWithNoInterface first;

    private final DependencyInterface second;



    @Autowired //tells spring which constructor to use for it's dependencies

    public MyStuff(DependencyWithNoInterface first, DependencyInterface second) {



        this.first = first;

        this.second = second;

    }



    public void run(){

        System.out.println("foo me");

        first.foo();

        second.superFoo();

    }

}


appContext.xml (seems you still need some XML)


<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xmlns:context="http://www.springframework.org/schema/context"

       xsi:schemaLocation="http://www.springframework.org/schema/beans

           http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

           http://www.springframework.org/schema/context

           http://www.springframework.org/schema/context/spring-context-3.0.xsd"
>

     <context:component-scan base-package="org.foo"/>

</beans>


Application.java (the initialization)


package org.foo;



import org.springframework.context.ApplicationContext;

import org.springframework.context.support.ClassPathXmlApplicationContext;



public class Application {

    public static void main(String args[]){

        ApplicationContext ctx =   new ClassPathXmlApplicationContext("appContext.xml"); //the IoC container

        MyStuff stff = ctx.getBean(MyStuff.class); //my fully injected class

        stff.run();

    }

}


 

Now I’m sure for the .NET developers familiar with StructureMap, Windsor, Autofac, etc this is completely unimpressive. I’m also sure there are Java developers that are somewhat unimpressed with this but for reasons that involve holding onto giant configuration artifacts. Manual IoC component registration falls where hand writing SQL for trivial data access does for me, extra repetitive work that has been solved years ago. I worked in factories a decade ago, and therefore now have a low tolerance for similar activity as a professional.

Edit:

It appears you can forgo XML as well in your main class use a different Application context and add the refresh and scan lines


package org.foo;



import org.springframework.context.annotation.AnnotationConfigApplicationContext;



public class Application {

    public static void main(String args[]){

        AnnotationConfigApplicationContext ctx =   new AnnotationConfigApplicationContext(); 

        ctx.scan("org.foo"); //scans the org.foo package

        ctx.refresh(); //needed to load them for some reason

        MyStuff stff = ctx.getBean(MyStuff.class);

        stff.run();

    }

}


Thursday, July 15, 2010

Anti-Pattern: Too much of your application is about interacting with external resources

Firstly, what am I talking about? Applications that meet some of the following descriptions:
  1. Stored procedures with a fair amount of conditional logic or complicated business rules buried in a sub-query (some would argue sprocs at all).
  2. Web pages with lots of conditional output again encoding business rules in snippets of view logic. if else checks that display the latest price for something if before and certain date or a default price.
  3. The majority of ‘unit tests’ require a database, or web and application server to be up and running.
  4. The majority of code files make reference to language default I/O or database libraries.
  5. Hours are spent trying to determine if differences in version of infrastructure are the cause of certain bugs.
  6. Changing database schema results in hours of refactoring, as hundreds of querys and sprocs are hunted through to see if this index or that index is being properly hit.
Why is this an anti-pattern? First before I bore you with endless reasoning behind SOLID principles, proper OO etc if you’re reading this and you’re of a bent that disagrees with me so far entirely..none of that will mean anything to you, and none of it will mean anything to your customers so let me put this in the most practical terms I can.
It is not testable in the slightest in any practical sense. You will be counter with well that with that sort of application and making everything a front to back test that you KNOW when things are working, yes but you RARELY if EVER know in a quick sense why things are not working. If i can reliably eliminate any chance that my actual code or business logic is the cause of a problem, if all of my fancy business rules are in something that can be run thousands of times a second, then I can throw all sorts of corner cases at my code base and not increase my verification time. If the majority of my application is business logic and plain old code then I can save my slow manual testing to the areas that are so bullet proof and only have a slice of my application doing front to back testing to yes, make sure things really work.
Note: Cross posted from Polyglots R Us.

Permalink

Saturday, July 10, 2010

Project Management in Java: A Confused .NET Developer’s Perspective

When I was first introduced to workplace Java the amount of ways one could define a project appeared to be restrictive, confusing and a point of frequent friction. While those things may all be true, it’s a great deal better than I would have expected since I was coming from a world where an upgrade to a newer version of Visual Studio requires your whole team to sync up to that same version!  So how do cross IDEA teams get any work done in the Java workplace? Do Java OSS projects require everyone uses Eclipse, NetBeans, or IntelliJ?  Does every new IDE need to have import/export filters for all of the other existing ones and if it becomes popular do the already present IDE’s have to include this? Which system is the best and what matters?
To answer these questions let break down what makes a project and while doing so compare and contrast the Java perspective with the .NET one.

Compilation

Java IDE’s  (Intellij/Eclipse/Netbeans) – basically run “make” on the fly and pass everything to javac.
JAVA IDE-agnostic project systems  (Maven, Gradle) – adding all the references contained directly to the classpath and then calling javac.
.NET (Visual Studio,MSBuild)  – MSBuild feeds all references to csc.
In summary, there is more similar than different here.

Strong Versioning

Java IDE’s – at least in the case of IntelliJ I ended up manually writing a MANIFEST.MF file.
Java IDE-agnostic project systems – still manual editing a lot of the time but usually just one line already filled out by your template that you edit.  I know NetBeans gives you a GUI to edit this.
.NET – GUI but you can manually edit an assembly.cs file.
In summary,  Java IDE’s at least in the case of IntelliJ really are using default Java by hand stuff.

Dependencies

Java IDE’s – All use convention! Basically, search for jar files in your source tree, if the IDE finds them it assumes you need them and adds them as references in your project format. Works pretty nice on a whole and allows for manual tweaking in the oddball cases.
Java IDE-agnostic project systems – requires explicit dependency reference by version, however they will download them automatically from remote locations and put them in your file-system in a specific way (defined by them unless overridden by you). Since both of these are also responsible for your compilation you have to basically run everything through them or your IDE has to be aware of them.
.NET – requires explicit reference to DLL’s on the file system.
In summary, .NET and the Java IDE-agnostic systems are VERY similar here in approach, whereas the Java IDE’s take a convention over configuration approach. I’d say there are times I definitely prefer the Java IDE approach and there are times the capabilities of the Java IDE-agnostic systems are worth it.

Source References

All Java solutions – do reference by directory structure and wild card matching.
.NET – explicit references to each class file .
In summary, this is the HUGE difference. Java takes more of a Make based approach where it’s EXPECTED this will not be the only IDE/project system ever used on this source tree, and this is where the Visual Studio/.NET experience starts to fall apart when trying to use cross-ide’s or versions. It has forced everyone to implement and keep up with MSBuild’s format to enable other IDE’s to be used, and don’t talk about actual alternatives to MSBuild. At this point even though things like NAnt and Rake/Albacore are more popular among agile .NET teams than scripting MSBuild almost all of those scripts still call MSBuild to do the compile step!

Questions Answered

Q. How do Java OSS projects deal with all the different IDE’s out there and even different project formats?/How do cross IDE teams get work done?
A. They don’t have to worry about it as much as .NET devs. A friend of mine who a lead contributor to the Apache Cassandra project in fact included a Maven file for the “enterprise types” but by and large use Ant’s to do their builds and he himself has used a number of IDE’s on that same codebase. .NET OSS development is still a bit hamstrung by MSBuild and really alternative IDE’s are keeping in sync with that project management format. There are real problems with style and code format guidelines, but there are usually workarounds or ways of solving this if code style guidelines is something you really care about ( which I don’t).
Q. Which system is the best?
A. There is no one size fits all answer here. I now know Apache Maven solves some problems for people and even though I don’t appreciate it’s approach to things, it’s done some valuable good for the community.  The IDE-centric approach really works pretty well a lot of the time for projects. Gradle, while being the closest to what I want is very young and has limited IDE support, at least it’s not easy enough for this Java n00b to figure out how to use with IntelliJ without some friction.

Summary

I hope that all wasn’t too confusing, it sure was for me to go through while I had all these options , and in a completely different environment to what I was used to. Worse a lot of the folks I talked to saw these IDE-agnostic project systems as solving problems they didn’t have. Maven, Ivy, etc was a world that was buzzword laden (I’m really looking at you Maven and Ivy), involved heavy setup time for relatively simple tasks, incomplete documentation (looking at you Gradle) and was frequently badly misused (those who know me know what I’m referring too here). In the end an IDE-centric approach in the Java world does not seem to tie you to that IDE in any way I can find, and for getting started had little friction, and for people new to Java it seemed to be the best way to get started. Besides in Java at least it’s easy to change your mind later.
Note: Cross posted from Polyglots R Us.

Permalink

Thursday, July 1, 2010

MySQL 5 Performance Tuning Toolkit

Recently we’d played with table partitioning and because of the limitations of it and some decisions we’d made a very long time ago we ended up spending a couple of days tracking down hotspots. In the process I picked up a few tricks, I’m not even an authority on any of this but I figured it’d help out others in the same spot.

Logging Toolkit

If you have a slow loading page or test and have no idea which one of the thirty or so queries going on is the cause how to do you find it.  Slow query log and logging queries that are not using indexes. Adding the following to your my.ini, my.cnf or whatever your mysql config file is named on your system and you’ll be able to get a list of offenders to further investigate.

MySQL Console Toolkit

SHOW INNODB STATUS – gives tons of useful information on what the current state of deadlocks, transactions and all sorts of information.
SHOW FULL PROCESSLIST – shows the current threads and queries running on them. running this over and over again can show you queries that are hanging.
EXPLAIN <QUERY>– Just in place of query type your full query statement, particularly in sub-queries, this is great for pointing out potential problems, in my most recent case I had an uncacheable sub-query, pointing out quickly something that was just not going to work no matter what.

Final Notes

Anyway this is only a smidgen of the useful stuff you need in your toolkit and I’m certainly no expert. I want to plug High Performance MySQL: Optimization, Backups, Replication, and More and the http://www.mysqlperformanceblog.com/ for it’s help and as a reference for those wanting to go into deeper study.
Note: Cross posted from Polyglots R Us.

Permalink

Saturday, June 26, 2010

Hibernate Connection Pooling: why isn't the default one for production?

Hibernate unlike NHibernate comes with a variety of connection pooling options. The three primary ones of which I'm aware are Proxool, Apache DBCP, and c3p0 . I myself have only so far used c3p0 and it works quite well having saved me from a couple of jams so far where the default one was in use previously. First why not use the default one? Because the official Hibernate documentation says in a big yellow box marked caution the following words:

The built-in Hibernate connection pool is in no way intended for production use. It lacks several features found on any decent connection pool.

Yet on almost every single Java project I've worked on the default is in use. I wouldn't care so much but this has personally wasted a fair amount of time for me and I've had to approach this as an outsider every time getting into the argument "it hasn't cause any issues before". Yet usually within a couple of days of that discussion I or someone else has resolved a "well it just does that sometimes" problem just by switching the connection pool. However, in implementing an alternate connection pool I should share a couple of gotcha's

1) Turn on logging for Hibernate and read the logs and make sure your connection pool of choice is actually activated and not the default one.

2) Older examples of the documentation I found for c3p0 did not include the necessary line for the C3P0Connection provider. If this line is not included in version 3.x version of hibernate regardless of what else is configured hibernate will use the default connection pool.

3) If the class you specified for connection provider line is not found for some reason, also the default connection pool will be used. Depending on your version of hibernate you may have to reference a different jar.

Below is the config borrowed from the hibernate connection tutorial only with c3p0 configured, please borrow it instead of using the default one.

<div class="line" id="LC1"><span class="cp"><?xml version='1.0' encoding='utf-8'?></span></div><div class="line" id="LC2"><span class="cp"><!DOCTYPE hibernate-configuration PUBLIC</span></div><div class="line" id="LC3"><span class="cp">        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"</span></div><div class="line" id="LC4"><span class="cp">        "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd"></span></div><div class="line" id="LC5"><br></div><div class="line" id="LC6"><span class="nt"><hibernate-configuration></span></div><div class="line" id="LC7"><br></div><div class="line" id="LC8">    <span class="nt"><session-factory></span></div><div class="line" id="LC9"><br></div><div class="line" id="LC10">        <span class="c"><!-- Database connection settings --></span></div><div class="line" id="LC11">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"connection.driver_class"</span><span class="nt">></span>org.hsqldb.jdbcDriver<span class="nt"></property></span></div><div class="line" id="LC12">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"connection.url"</span><span class="nt">></span>jdbc:hsqldb:hsql://localhost<span class="nt"></property></span></div><div class="line" id="LC13">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"connection.username"</span><span class="nt">></span>sa<span class="nt"></property></span></div><div class="line" id="LC14">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"connection.password"</span><span class="nt">></property></span></div><div class="line" id="LC15">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"connection.provider_class"</span><span class="nt">></span>org.hibernate.connection.C3P0ConnectionProvider<span class="nt"></property></span> <span class="c"><!-- NECESSARY TO USE C3P0 --></span></div><div class="line" id="LC16">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.acquire_increment"</span><span class="nt">></span>1<span class="nt"></property></span> </div><div class="line" id="LC17">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.idle_test_period"</span><span class="nt">></span>100<span class="nt"></property></span> <span class="c"><!-- seconds --></span> </div><div class="line" id="LC18">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.max_size"</span><span class="nt">></span>100<span class="nt"></property></span> </div><div class="line" id="LC19">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.max_statements"</span><span class="nt">></span>0<span class="nt"></property></span> </div><div class="line" id="LC20">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.min_size"</span><span class="nt">></span>10<span class="nt"></property></span> </div><div class="line" id="LC21">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"c3p0.timeout"</span><span class="nt">></span>100<span class="nt"></property></span> <span class="c"><!-- seconds --></span> </div><div class="line" id="LC22">        <span class="c"><!-- SQL dialect --></span></div><div class="line" id="LC23">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"dialect"</span><span class="nt">></span>org.hibernate.dialect.HSQLDialect<span class="nt"></property></span></div><div class="line" id="LC24"><br></div><div class="line" id="LC25">        <span class="c"><!-- Enable Hibernate's automatic session context management --></span></div><div class="line" id="LC26">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"current_session_context_class"</span><span class="nt">></span>thread<span class="nt"></property></span></div><div class="line" id="LC27"><br></div><div class="line" id="LC28">        <span class="c"><!-- Disable the second-level cache  --></span></div><div class="line" id="LC29">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"cache.provider_class"</span><span class="nt">></span>org.hibernate.cache.NoCacheProvider<span class="nt"></property></span></div><div class="line" id="LC30"><br></div><div class="line" id="LC31">        <span class="c"><!-- Echo all executed SQL to stdout --></span></div><div class="line" id="LC32">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"show_sql"</span><span class="nt">></span>true<span class="nt"></property></span></div><div class="line" id="LC33"><br></div><div class="line" id="LC34">        <span class="c"><!-- Drop and re-create the database schema on startup --></span></div><div class="line" id="LC35">        <span class="nt"><property</span> <span class="na">name=</span><span class="s">"hbm2ddl.auto"</span><span class="nt">></span>update<span class="nt"></property></span></div><div class="line" id="LC36"><br></div><div class="line" id="LC37">        <span class="nt"><mapping</span> <span class="na">resource=</span><span class="s">"org/hibernate/tutorial/domain/Event.hbm.xml"</span><span class="nt">/></span></div><div class="line" id="LC38"><br></div><div class="line" id="LC39">    <span class="nt"></session-factory></span></div><div class="line" id="LC40"><br></div><div class="line" id="LC41"><span class="nt"></hibernate-configuration></span></div>

Sunday, June 6, 2010

Dynamic DNS with Amazon EC2 Linux and EveryDNS

So I finally sat down and did the math and found out Amazon EC2 was quite a bit cheaper than what I’d been paying for hosting as long as I was willing to prepay for at least a year.  However, with EC2 you are getting a dynamic ip’s so what to do? Well I’ve been using EveryDNS for years for my dns hosting and despite it’s recent acquisition by Dyn, Inc it’s still works the same. The scripts that are described below can be downloaded here, what follows is a brief walkthrough.

  1. Download EveryDns update script from http://www.everydns.com/dynamic.php  and place it in something like /usr/local/bin.
  2. Create a convenience script in /usr/local/bin that will get your external ip and have it contain the following lines and call it external-ip.sh . This is using Amazon’s suggested curl command to find out your EC2 instance Ip address.
    1. external-ip
  3. Then create a script for the actual dynamic DNS update in /usr/local/bin/ and call it updatedns-sh. This is using the set command to grab the output of external ip and storing it in the variable $1, and then passing that value to the update command. NOTE: this will update the dns record of ALL your domains with everydns set to dynamic. If you need to use specific domain names there is a –d flag for the eDNS client.update
  4. create a cron job on your linux distro of choice but the cron job line should look like the following.
    1. cron

Saturday, May 22, 2010

Projects in Java with Maven 2

 

For those of you who don’t know Maven is a build tool/dependency manager/project model. Those in the Microsoft space can probably imagine MSBuild + the ability to download all dll’s for you.

What I liked

  1. Dependency Resolution. Automatic downloading of dependencies rocks in concept. Just specify library name and version and when you compile again it’s there.  No need to check in jar’s into your source tree.
  2. IDE Independence. In practical terms lets you use whatever IDE you want with no import/export.  Intellij, Eclipse and Netbeans all understand maven as a full project format. So your team can all have different IDE’s and not create havoc with one another.
  3. Good Default Project Structure. Tests are in the same location by default, resources for tests and your prod code are in expected places.

What I hated

  1. Inefficient with already downloaded dependencies. It will check remote repositories every time you build even when you specified version number. Now I can see why they did that but not as a default. I mean sure version 1.12 may have had a bad bug and the project hotfixed in a new one with the same version number, but I’d say that’s really unlikely.  In a larger project with several repositories the difference in time to build between this and an alternate dependency manager Apache Ivy is stark.
  2. Very opinionated view of the build process. Those of you used to (N)Ant or Rake will miss the lack of control. The AntRun plugin will help mitigate some of the lose of control.
  3. HOME/.m2/settings.xml . Specific system wide settings do not belong in a build language. The bad heavy friction ideas that it enabled were legion. Just one example was I had to connect to my work VPN even when working on my home OSS projects hosted on github.
  4. Way too XML for for sometimes simple things. To specify the version of java target language version see the figure below, this specifies Java 1.6.

 


<build>

  <plugins>

    

    <plugin>

      <artifactId>maven-compiler-plugin</artifactId>

      <configuration>

        <source>1.6</source>

        <target>1.6</target>

      </configuration>

    </plugin>

    

  </plugins>

</build>


Summary

If you have a cross IDE team or are running an OSS java project I find it very hard to ignore Maven. If you are looking for easier dependency management, less verbose build files, and do not care about IDE independence I suggest any number of alternatives such as Apache Buildr, Apache Ivy, Gradle or many others I’m sure I’m forgetting.