Wednesday, December 28, 2005

LOLA & FLOSS

If you've never heard of the Law of Leaky Abstractions (LOLA) then you ought to read Joel Spolsky's blog on the topic.

So what does LOLA have to do with FLOSS? Everything, because if you cut code FLOSS makes it orders of magnitude easier to deal with LOLA and her mood swings. Case in point, what is wrong with following innocent lines of code?

File destination = new File( "somedestination" );
File source = new File( "somesource" );
FileChannel from = new FileInputStream( source ).getChannel();
try
{
    FileChannel to = new FileOutputStream( destination ).getChannel();
    try
    {
        for( long pos = 0, size = from.size(); pos < size; )
            pos += from.transferTo( pos, size - pos, to );
    }
    finally
    {
        to.close();
    }
}
finally
{
   from.close();
}

Well ... any ideas?

The correct answer is supposed to be nothing but if the previous lines of code are running in a Sun Microsystems' JVM on a Linux box with a 2.6.x kernel with a version of the Linux CIFS client that supports the directio flag and source or destination is on a Windows share mounted with directio then LOLA will leak the file abstraction all over my pristine code. Line 9 will throw an IOException with the message: "device does not exist".

Let me walk you through the steps of trying to figure out what the IOException means. My first step in tracking this down would be to follow through with what the error message is telling me. So I checked that the source file did indeed exist and that the process has the appropriate permissions to read from it. Next, I checked that the destination directory existed and that the process has the appropriate permissions to write to it. I even went as far as creating the destination file manually. Everything worked. Executed the code again ... boom, it blew up again. Obviously something is wrong with the code so now it was time to dig into the guts of it to figure out what is going on. Now imagine a FLOSSless world where I have zero access to the source code of the libraries upon which my program depends. I wouldn't be able to go any further. LOLA would have kicked my ass and that would be that or I would have to decompile the class files and run the risk of getting sued by the vendor. Fortunately I don't have that problem. All I have to do from my IDE (IntelliJ/IDEA) is right click on the transferTo method on line 9, select Go To -> Implementation and start working my way through the code.

So what's the problem? It turns out that on Sun JVMs for file-to-file transfer FileChannel.transfer(From|To) likes to use memory mapping (mmap) instead of the sendfile system call (zero-copy). The problem with mmap is shares mounted with directio doesn't support memory mapping which is why we end up with a "device does not exist" IOException. My solution was simply to fall back to good old fashion byte[] copying. Booyah! Another problem solved thanx to FLOSS.

Solution:

File destination = new File( "somedestination" );
File source = new File( "somesource" );
FileInputStream srcStream = new FileInputStream( source );
FileChannel from = srcStream.getChannel();
try
{
    FileOutputStream dstream = new FileOutputStream( destination );
    FileChannel to = dstream.getChannel();
    try
    {
        for( long pos = 0, size = from.size(); pos < size; )
            pos += from.transferTo( pos, size - pos, to );
    }
    catch( IOException e )
    {   // Just in case it failed because of something else.  
        if( to.position() > 0 )
            throw new IOException( e.toString() );
        byte[] bytes = new byte[ (int)(Math.min( Bytes.KILOBYTE << 6, source.length() )) ];
        for( int eof; -1 != (eof = srcStream.read( bytes )); )
            dstream.write( bytes, 0, eof );
    }
    finally
    {
        to.close();
    }
}
finally
{
    from.close();
}

Wednesday, October 05, 2005

Mustang is comming, it's time to face the Tiger

So you have been waiting patiently for the the dust settle, the bugs to rear their ugly heads and get squashed, the PR machine to choke on it's own shite and die, the {tech articles, books, and blogs} written, your favorite IDE to support it and finally the name of the damn thing to stabilize.

Aside:

Since I've milked all the humor I can out of Sun's marketing fiasco with the naming of the Tiger release I'm going to put it to rest and refer to Tiger either as Tiger or Java 1.5. No more snide remarks and no more conspiracy theories about the marketing department's Pinky and the Brain like attempt to rule the world by creating mass panic and confusion across the land by first naming it 1.5 then changing it to 5.0. Causing the masses to think some unknown quantity of time had mysteriously evaporated between the transition from 1.5 to 5.0, a jump of 3.5 releases almost instantaneously. The Brain, err ...Sun would then step in explaining that the earth had passed through a worm hole and during the lost time Bill/Microsoft had been defeated by Scott/Sun and Sun was now the 800lb gorilla thus all is well and proves once and for all that Stanford graduates are smarter than Harvard drop-outs.

So guess what? ..... It's all come to pass and it's time to upgrade to Tiger. There is absolutely no reason to be a holdout. There is just too many goodies in it not to risk the move. But, if all you have to go on is the what Sun's PR machine had been churning out you might think the only thing good in Tiger is:

  1. autoboxing
  2. varags
  3. annotations
  4. static imports
  5. type-safe enums
  6. enhanced for loop

Rant:

I was going to include generics in my list but I decided against it. The list as it exists, reflect the so called simplification of Java features and anyone that thinks Java's generics implementation is a simplification is a complete moron. The fact that for anything but the simplest case you have to redeclare the inheritance hierarchy to use it says it all. I'm not even going to spit venom on the fact that it's impossible to create a generic array or how all that wonderful generic type information that they force you to finger fu*k into your source code (have you seen the syntax? Nuff said!) suddenly disappears at runtime. But I'm not bitter!

Though as far as I'm concerned the only thing worthwhile on that list is 5 and 6. So now that we have gotten pass "the reasons for VB programmers to jump to Java" lets talk about the stuff that's important to me.

Firstly they went back and tuned up the memory model. This to me is the most important thing that Sun got right in Tiger because it solves once and for all the inconsistencies related to multi threaded programming in Java and if you are a reader of my blog you ought to know how important a topic I think multi threaded programming is.

They gave us java.util.concurrent, java.util.concurrent.atomic, and java.util.concurrent.locks. These three packages combine to make it possible to write scalable multi threaded applications. There is enough there to act as the foundation for frameworks designed to simplify the task of writing multi threaded programs (i.ei.e. CSP) but abstractions can and do leak so it's important to know how all this stuff really works so that when something goes wrong you won't be sitting around with your thumb up your butt because you haven't the slightest clue of how things work under the covers.

Uncontended synchronization is cheap. It's so cheap that even though I am an advocate for java.util.concurrent I still use synchronized in some places. But I'm not entirely being fair to synchronized. It does have other benefits beside locking. It also enforces ordering, so it still plays an important role where execution ordering is important. Then again .... the ordering that synchronized provides can be achieved without using the synchronized keyword thanks to the additional semantics of the volatile keyword imposed by the updated memory model.

Finally, they have continued to improve object allocation performance. As a matter of fact, creating a new object is damned cheap especially if the heap is big enough. Object deallocation has also improved. In some cases it's free (more on this later).

So now the Java engineers over at Sun are working on Mustang (J2SE 1.6). This version is being billed as the desktop release and you are going to hear all about the cool new desktop features. So if you are GUI/AWT/Swing programmer there should be enough in there to get you moist maybe even wet. Given that I haven't written a Java GUI since I first learned Java and I don't plan on writing any anytime soon, I'm going to have to a take Sun's word for it . But there other things in Mustang that floats my boat. The most interesting one is that I can hack on it have my work included as part of the J2SE public release.

Aside:

I have mixed feelings about this. Firstly, I love Java so getting a direct hand in improving what I think is wrong with it and being able to share that with all the other bean-slingers is very appealing. But on the other hand, I'm not a big fan of Sun Microsystems, specifically their management staff (McNeally should be fired) and the stuff that comes out of their collective mouths. Talk about a confused message. They have put their foot in their mouths, McNeally and Schwartz especially, too many times for me to feel great about working for Sun for free because no matter what I do it will ultimately belong to Sun. Whenever they do something I agree with I can't help feeling it happened by accident or it was because the market forced them to and not because they had the foresight to do the right thing, i.e. OpenOffice.

But that's not all. One of the more encouraging things about Java is that Java the platform (JVM to be specific) continues to improve and remains at the center of a lot of research. That trend continues in Mustang. Read this. It was reading this that triggered my writing this entry in the first place. On the library side of things java.util.concurrent will sport a new ConcurrentMap built on a relatively new datastructure called skiplist. Though the jury is still out on whether skiplists are better (speed/memory) than B-Tree datastructrues like Red-Black Trees (i.e. java.util.TreeMap) and AVL trees. There are a couple of worthy JSRs that may make it into Mustang, the most worthy being this.

All in all Tiger and Mustang have a lot to offer programmers so if you are still a holdout it's time to let go and take the plunge.

Friday, June 17, 2005

The Free Lunch is Over

That's (part of) the title of an article I just finished reading describing the next revolution in computer programming. It fully supports the gist of a previous blog entry explaining the importance of parts of Java 1.5 (or Java 5.0 or whatever those marketing morons at Sun feel like calling it today). The article simply reinforces my assertion that the Java programming language is the place to be and that it will continue to be relevant for a long time to come.

Friday, February 25, 2005

Why java.util.concurrent Kicks Ass and Takes Names

There are many things that JDK 1.5 [or Java 2 version 5 or Java 5 or Java 2 version 1.5 the entire marketing department should be the ones fired in the lastest round of layoffs from Sun because of their malicious and spiteful attempt to confuse the general public with f*cked up product names Edition] got horribly wrong, (generics, static imports, etc) but there are a few things that they got really right and I'm going to spend some time talking about some of my favorites and why it's important to my fellow bean slinger.

Besides fixing the broken memory model, which laid the ground work for some of the stuff we'll be taking about, the bumbling morons at Sun (I'm still bitter about WebServices, EJBs, and Corba to name a few) finally added something to Java I completely agree with and is worth its weight in Internet porn. This my friends is the java.util.concurrent package. Some of my new favorite things about Java are located in this package and its children. But before I get into the nitty gritty let me paint the big picture for you. So bear with me as we take a journey through my warped view of time and space and the events therein.

One of the coolest and more useful features of the Java language has been the simplicity with which Java allows its masters to write multithreaded applications. Until recently it's been damn hard to write kick ass multithreaded applications for everyday use for [at least] two reasons. This is not to say there are no kick ass multithreaded apps out there. Truth is, just about every Java application is multithreaded out of the box but [the bigger point I'm trying to make is] they are ham stringed by single processor deployments (reason #1). Which means, you can launch all the threads you want but if you have only one processor to execute them on you are not going to get too far over a single threaded application. As a matter of fact, you are probably going to make things worse. The other problem is, writing great performing multithreaded applications is just damn hard (reason #2). This problem isn't specific to Java but Java never made it any easier and in some ways made it worse. This is because of the lack of synchronization primitives that Java exposed/provided and the ones that it did (synchronized, volatile) had some serious problems, especially for the uninitiated. Finally, it's just plain old damn hard to write great performing, deadlock free, starvation free, race condition free, multithread applications (reason #2 prime). Unless of course, your application is grossly parallel by nature, which very few applications are. So what has changed and what is changing that has brought me out of my shell to evangelize Java 1.5? Let's start with what is changing.

Multi-core CPUs are coming to a desktop near you and it won't cost you both your arms, or your legs, or your first born. Think about how easy it is to buy a dual processor desktop/workstation today. Now imagine 6 months to a year from now buying that same dual processor workstation, but instead of 2 processors you really have 4. Are you excited yet!?! Do you see where I'm going with this!?! Huh? Huh? Huh? Imagine users plugging their ears because your Java application that used to crawl when running on a single processor system is now screaming like a banshee [with a plasma hot piece of metal shoved in places where the sun don't shine] on a 4 way processor system. Great! Now lets talk about what has changed.

Simultaneous multithreading processors are here now, actually they have been around a while (since 2002), it's just that now they are a lot more accessible. Intel's HyperThreading technology is the poster child for affordable simultaneous multithreading CPUs. IBM's Power series [especially Power 5] is pretty ass kicking and name talking too, it's just expensive as hell. With the Intel stuff you can go out and buy one right now without getting kicked out of your apartment cause you just spent the rent money or have your house foreclosed on unless, you are a college student, or just out of college, or still living with your mother, or recently divorced and she/he took everything, or just got fired and you need the money to buy a gun. What this all adds up to is the capability to write ass kicking, name taking, multithreaded applications today.

So now you are convinced, excited, and ready to cut code like a cook chopping onions for Carbonnade a la Flamande. As a matter of fact, there is a tear in your eye not from the onions but because you are so happy about the potential that lays before you. Well, I'm going to give you the opportunity to wipe your eye and blow your nose because I'm stopping here. We'll pick this up again later when I've got more free time, cause right now, to keep myself from being kicked out of my apartment and not get fired. I've got to get back to work.