Wednesday, December 28, 2005

LOLA & FLOSS

If you've never heard of the Law of Leaky Abstractions (LOLA) then you ought to read Joel Spolsky's blog on the topic.

So what does LOLA have to do with FLOSS? Everything, because if you cut code FLOSS makes it orders of magnitude easier to deal with LOLA and her mood swings. Case in point, what is wrong with following innocent lines of code?

File destination = new File( "somedestination" );
File source = new File( "somesource" );
FileChannel from = new FileInputStream( source ).getChannel();
try
{
    FileChannel to = new FileOutputStream( destination ).getChannel();
    try
    {
        for( long pos = 0, size = from.size(); pos < size; )
            pos += from.transferTo( pos, size - pos, to );
    }
    finally
    {
        to.close();
    }
}
finally
{
   from.close();
}

Well ... any ideas?

The correct answer is supposed to be nothing but if the previous lines of code are running in a Sun Microsystems' JVM on a Linux box with a 2.6.x kernel with a version of the Linux CIFS client that supports the directio flag and source or destination is on a Windows share mounted with directio then LOLA will leak the file abstraction all over my pristine code. Line 9 will throw an IOException with the message: "device does not exist".

Let me walk you through the steps of trying to figure out what the IOException means. My first step in tracking this down would be to follow through with what the error message is telling me. So I checked that the source file did indeed exist and that the process has the appropriate permissions to read from it. Next, I checked that the destination directory existed and that the process has the appropriate permissions to write to it. I even went as far as creating the destination file manually. Everything worked. Executed the code again ... boom, it blew up again. Obviously something is wrong with the code so now it was time to dig into the guts of it to figure out what is going on. Now imagine a FLOSSless world where I have zero access to the source code of the libraries upon which my program depends. I wouldn't be able to go any further. LOLA would have kicked my ass and that would be that or I would have to decompile the class files and run the risk of getting sued by the vendor. Fortunately I don't have that problem. All I have to do from my IDE (IntelliJ/IDEA) is right click on the transferTo method on line 9, select Go To -> Implementation and start working my way through the code.

So what's the problem? It turns out that on Sun JVMs for file-to-file transfer FileChannel.transfer(From|To) likes to use memory mapping (mmap) instead of the sendfile system call (zero-copy). The problem with mmap is shares mounted with directio doesn't support memory mapping which is why we end up with a "device does not exist" IOException. My solution was simply to fall back to good old fashion byte[] copying. Booyah! Another problem solved thanx to FLOSS.

Solution:

File destination = new File( "somedestination" );
File source = new File( "somesource" );
FileInputStream srcStream = new FileInputStream( source );
FileChannel from = srcStream.getChannel();
try
{
    FileOutputStream dstream = new FileOutputStream( destination );
    FileChannel to = dstream.getChannel();
    try
    {
        for( long pos = 0, size = from.size(); pos < size; )
            pos += from.transferTo( pos, size - pos, to );
    }
    catch( IOException e )
    {   // Just in case it failed because of something else.  
        if( to.position() > 0 )
            throw new IOException( e.toString() );
        byte[] bytes = new byte[ (int)(Math.min( Bytes.KILOBYTE << 6, source.length() )) ];
        for( int eof; -1 != (eof = srcStream.read( bytes )); )
            dstream.write( bytes, 0, eof );
    }
    finally
    {
        to.close();
    }
}
finally
{
    from.close();
}