Implementing the Downloader
I am currently implementing a downloader into p300. This may sound easy since the HTTP-Server part of p300 is already written, but since I want to implement multi source downloading (and later swarming/partial file sharing) it is a bit more complicated.
My original goal was to use fixed-size chunks of 256 KB and request each chunk after another with a single HTTP Request. After having received a complete chunk, I store it in the output file (I am using memory mapped files here). I was using a new thread for each request, the request itself is done via a java.net.URLConnection object. I had to sacrifice Keep-Alive connections here because of some bug in the Java libraries (this was some months ago, I had to pause development until a few days ago).
This worked well with bandwidths in my VPN but drives my CPU insane when using it on a LAN, both on the server-side p300 and on the client-side p300. So I changed both sides to use a thread pool and this made it a bit better, but not optimal. Also, if you want to achieve high bandwidths with TCP, you need to have it running for some time so the send and receive windows will grow. The bandwidth with my approach was not higher than 300 KB/sec, caused because of the re-Negotiation of the TCP-Connection and the HTTP-Request.
To make it short: my current approach of receiving (small) chunks sequentially did not work out.
I thought of multiple things, inclung HTTP Pipelining (which does not play well with URLConnection i suppose) and increasing the chunk size or sacrificing chunking altogether but I think I have a solution that requires only minimal changes:
Instead of requesting a single chunk, I request from the chunk start to the end of the file. Then I read all the bytes, and everytime I am on a (virtual) chunk boundary I check if we already have that chunk. If yes, disconnect, if no, continue and mark the previous/current chunk as loaded.
I’ll try to do this in the next days (No, this does not take days, but I don’t have much time for it each day).
I hope it works out 
(Part 2)
