Release 0.4.5

code markdown for commands and paths in README.md

README.md

@@ -4,26 +4,26 @@ MuWire is an easy to use file-sharing program which offers anonymity using [I2P
 
 It is inspired by the LimeWire Gnutella client and developped by a former LimeWire developer.
 
-The current stable release - 0.2.5 is avaiable for download at http://muwire.com.  You can find technical documentation in the "doc" folder.
+The current stable release - 0.4.0 is avaiable for download at https://muwire.com.  You can find technical documentation in the "doc" folder.
 
 ### Building
 
 You need JRE 8 or newer.  After installing that and setting up the appropriate paths, just type
 
 ```
-./gradlew assemble 
+./gradlew clean assemble 
 ```
 
 If you want to run the unit tests, type
 ```
-./gradlew build
+./gradlew clean build
 ```
 
 Some of the UI tests will fail because they haven't been written yet :-/
 
 ### Running
 
-You need to have an I2P router up and running on the same machine.  After you build the application, look inside "gui/build/distributions".  Untar/unzip one of the "shadow" files and then run the jar contained inside by typing "java -jar MuWire-x.y.z.jar" in a terminal or command prompt.  If you use a custom I2CP host and port, create a file $HOME/.MuWire/i2p.properties and put "i2cp.tcp.host=<host>" and "i2cp.tcp.port=<port>" in there.
+You need to have an I2P router up and running on the same machine.  After you build the application, look inside `gui/build/distributions`.  Untar/unzip one of the `shadow` files and then run the jar contained inside by typing `java -jar MuWire-x.y.z.jar` in a terminal or command prompt.  If you use a custom I2CP host and port, create a file `$HOME/.MuWire/i2p.properties` and put `i2cp.tcp.host=<host>` and `i2cp.tcp.port=<port>` in there.
 
 The first time you run MuWire it will ask you to select a nickname.  This nickname will be displayed with search results, so that others can verify the file was shared by you.  It is best to leave MuWire running all the time, just like I2P.
 
@@ -31,3 +31,26 @@ The first time you run MuWire it will ask you to select a nickname.  This nickna
 ### Known bugs and limitations
 
 * Many UI features you would expect are not there yet
+
+### Quick FAQ
+
+* why is MuWire slow ?
+  
+  - too few sources you're downloading from
+  - you can increase the number of tunnels by using more tunnels via Options->I2P Inbound/Outbound Quantity 
+    the default is 4 and you could raise up to as high as 16 ( Caution !!!!)
+
+* my search is not returning (enough) results !
+
+  - search is keyword or hash based
+  - keywords and hash(es) are NOT regexed or wildcarded so they have to be complete
+    so searching for 'musi' will not return results with 'music' - you have to search for 'music'
+  - ALL keywords have to match 
+  - only use space for keyword separation
+  - if you already have the file in question it is not displayed ( can be changed via Options )
+
+* what's this right click -> 'Copy hash to clipboard' for ?
+
+  - if you have a specific file you wish to share or download you can use the hash as a unique identifier
+    to make sure you have exactly the right file.
+  - you can share this hash with others to ensure they are getting the right file

TODO.md

@@ -4,10 +4,6 @@ Not in any particular order yet
 
 ### Big Items
 
-##### Alternate Locations
-
-This helps peers discover new sources for a file while the download is in progress.  Also makes sharing of partial files possible.
-
 ##### Bloom Filters
 
 This reduces query traffic by not sending last hop queries to peers that definitely do not have the file
@@ -24,10 +20,6 @@ For helping users make better decisions whom to trust
 
 To allow every user to not route queries for content they do not like.  This is mostly GUI work, the backend part is simple
 
-##### Packaging With JRE, Embedded Router
-
-For ease of deployment for new users, and so that users do not need to run a separate I2P router
-
 ##### Web UI, REST Interface, etc.
 
 Basically any non-gui non-cli user interface
@@ -38,8 +30,6 @@ To enable parsing of metadata from known file types and the user editing it or a
 
 ### Small Items
 
-* Detect if router is dead and show warning or exit
 * Wrapper of some kind for in-place upgrades
 * Download file sequentially
-* Unsharing of files
 * Multiple-selection download, Ctrl-A

cli/src/main/groovy/com/muwire/cli/Cli.groovy

@@ -35,7 +35,7 @@ class Cli {
         
         Core core 
         try {
-            core = new Core(props, home, "0.2.9")
+            core = new Core(props, home, "0.4.5")
         } catch (Exception bad) {
             bad.printStackTrace(System.out)
             println "Failed to initialize core, exiting"

cli/src/main/groovy/com/muwire/cli/CliDownloader.groovy

@@ -53,7 +53,7 @@ class CliDownloader {
 
         Core core
         try {
-            core = new Core(props, home, "0.2.9")
+            core = new Core(props, home, "0.4.5")
         } catch (Exception bad) {
             bad.printStackTrace(System.out)
             println "Failed to initialize core, exiting"

core/build.gradle

@@ -2,6 +2,7 @@ apply plugin : 'application'
 mainClassName = 'com.muwire.core.Core'
 applicationDefaultJvmArgs = ['-Djava.util.logging.config.file=logging.properties']
 dependencies {
+    compile 'net.i2p:router:0.9.40'
     compile 'net.i2p.client:mstreaming:0.9.40'
     compile 'net.i2p.client:streaming:0.9.40'
 

core/src/main/groovy/com/muwire/core/Constants.groovy

@@ -9,7 +9,5 @@ class Constants {
     public static final int MAX_HEADER_SIZE = 0x1 << 14
     public static final int MAX_HEADERS = 16
     
-    public static final float DOWNLOAD_SEQUENTIAL_RATIO = 0.8f
-    
-    public static final String SPLIT_PATTERN = "[\\.,_-]"
+    public static final String SPLIT_PATTERN = "[\\*\\+\\-,\\.:;\\(\\)=_/\\\\\\!\\\"\\\'\\\$%\\|\\[\\]\\{\\}\\?]"
 }

core/src/main/groovy/com/muwire/core/Core.groovy

@@ -1,6 +1,7 @@
 package com.muwire.core
 
 import java.nio.charset.StandardCharsets
+import java.util.concurrent.atomic.AtomicBoolean
 
 import com.muwire.core.connection.ConnectionAcceptor
 import com.muwire.core.connection.ConnectionEstablisher
@@ -12,8 +13,11 @@ import com.muwire.core.connection.I2PConnector
 import com.muwire.core.connection.LeafConnectionManager
 import com.muwire.core.connection.UltrapeerConnectionManager
 import com.muwire.core.download.DownloadManager
+import com.muwire.core.download.SourceDiscoveredEvent
 import com.muwire.core.download.UIDownloadCancelledEvent
 import com.muwire.core.download.UIDownloadEvent
+import com.muwire.core.download.UIDownloadPausedEvent
+import com.muwire.core.download.UIDownloadResumedEvent
 import com.muwire.core.files.FileDownloadedEvent
 import com.muwire.core.files.FileHashedEvent
 import com.muwire.core.files.FileHasher
@@ -23,15 +27,19 @@ import com.muwire.core.files.FileSharedEvent
 import com.muwire.core.files.FileUnsharedEvent
 import com.muwire.core.files.HasherService
 import com.muwire.core.files.PersisterService
+import com.muwire.core.files.AllFilesLoadedEvent
+import com.muwire.core.files.DirectoryUnsharedEvent
 import com.muwire.core.files.DirectoryWatcher
 import com.muwire.core.hostcache.CacheClient
 import com.muwire.core.hostcache.HostCache
 import com.muwire.core.hostcache.HostDiscoveredEvent
+import com.muwire.core.mesh.MeshManager
 import com.muwire.core.search.QueryEvent
 import com.muwire.core.search.ResultsEvent
 import com.muwire.core.search.ResultsSender
 import com.muwire.core.search.SearchEvent
 import com.muwire.core.search.SearchManager
+import com.muwire.core.search.UIResultBatchEvent
 import com.muwire.core.trust.TrustEvent
 import com.muwire.core.trust.TrustService
 import com.muwire.core.update.UpdateClient
@@ -45,6 +53,7 @@ import net.i2p.client.I2PSession
 import net.i2p.client.streaming.I2PSocketManager
 import net.i2p.client.streaming.I2PSocketManagerFactory
 import net.i2p.client.streaming.I2PSocketOptions
+import net.i2p.client.streaming.I2PSocketManager.DisconnectListener
 import net.i2p.crypto.DSAEngine
 import net.i2p.crypto.SigType
 import net.i2p.data.Destination
@@ -52,6 +61,9 @@ import net.i2p.data.PrivateKey
 import net.i2p.data.Signature
 import net.i2p.data.SigningPrivateKey
 
+import net.i2p.router.Router
+import net.i2p.router.RouterContext
+
 @Log
 public class Core {
     
@@ -73,13 +85,63 @@ public class Core {
     private final DownloadManager downloadManager
     private final DirectoryWatcher directoryWatcher
     final FileManager fileManager
+    final UploadManager uploadManager
+    
+    private final Router router
+    
+    final AtomicBoolean shutdown = new AtomicBoolean()
         
     public Core(MuWireSettings props, File home, String myVersion) {
         this.home = home		
         this.muOptions = props
-        log.info "Initializing I2P context"
-        I2PAppContext.getGlobalContext().logManager()
-        I2PAppContext.getGlobalContext()._logManager = new MuWireLogManager()
+        
+        i2pOptions = new Properties()
+        def i2pOptionsFile = new File(home,"i2p.properties")
+        if (i2pOptionsFile.exists()) {
+            i2pOptionsFile.withInputStream { i2pOptions.load(it) }
+            
+            if (!i2pOptions.containsKey("inbound.nickname"))
+                i2pOptions["inbound.nickname"] = "MuWire"
+                if (!i2pOptions.containsKey("outbound.nickname"))
+                    i2pOptions["outbound.nickname"] = "MuWire"
+        } else {
+            i2pOptions["inbound.nickname"] = "MuWire"
+            i2pOptions["outbound.nickname"] = "MuWire"
+            i2pOptions["inbound.length"] = "3"
+            i2pOptions["inbound.quantity"] = "4"
+            i2pOptions["outbound.length"] = "3"
+            i2pOptions["outbound.quantity"] = "4"
+            i2pOptions["i2cp.tcp.host"] = "127.0.0.1"
+            i2pOptions["i2cp.tcp.port"] = "7654"
+            Random r = new Random()
+            int port = r.nextInt(60000) + 4000
+            i2pOptions["i2np.ntcp.port"] = String.valueOf(port)
+            i2pOptions["i2np.udp.port"] = String.valueOf(port)
+            i2pOptionsFile.withOutputStream { i2pOptions.store(it, "") }
+        }
+        
+        if (!props.embeddedRouter) {
+            log.info "Initializing I2P context"
+            I2PAppContext.getGlobalContext().logManager()
+            I2PAppContext.getGlobalContext()._logManager = new MuWireLogManager()
+            router = null
+        } else {
+            log.info("launching embedded router")
+            Properties routerProps = new Properties()
+            routerProps.setProperty("i2p.dir.config", home.getAbsolutePath())
+            routerProps.setProperty("router.excludePeerCaps", "KLM")
+            routerProps.setProperty("i2np.inboundKBytesPerSecond", String.valueOf(props.inBw))
+            routerProps.setProperty("i2np.outboundKBytesPerSecond", String.valueOf(props.outBw))
+            routerProps.setProperty("i2cp.disableInterface", "true")
+            routerProps.setProperty("i2np.ntcp.port", i2pOptions["i2np.ntcp.port"])
+            routerProps.setProperty("i2np.udp.port", i2pOptions["i2np.udp.port"])
+            routerProps.setProperty("i2np.udp.internalPort", i2pOptions["i2np.udp.port"])
+            router = new Router(routerProps)
+            I2PAppContext.getGlobalContext().metaClass = new RouterContextMetaClass()
+            router.runRouter()
+            while(!router.isRunning())
+                Thread.sleep(100)
+        }
         
 		log.info("initializing I2P socket manager")
 		def i2pClient = new I2PClientFactory().createClient()
@@ -91,25 +153,6 @@ public class Core {
 			}
 		}
 		
-        i2pOptions = new Properties()
-        def i2pOptionsFile = new File(home,"i2p.properties")
-        if (i2pOptionsFile.exists()) {
-            i2pOptionsFile.withInputStream { i2pOptions.load(it) }
-
-            if (!i2pOptions.containsKey("inbound.nickname"))
-                i2pOptions["inbound.nickname"] = "MuWire"
-            if (!i2pOptions.containsKey("outbound.nickname"))
-                i2pOptions["outbound.nickname"] = "MuWire"
-        } else {
-            i2pOptions["inbound.nickname"] = "MuWire"
-            i2pOptions["outbound.nickname"] = "MuWire"
-            i2pOptions["inbound.length"] = "3"
-            i2pOptions["inbound.quantity"] = "2"
-            i2pOptions["outbound.length"] = "3"
-            i2pOptions["outbound.quantity"] = "2"
-            i2pOptions["i2cp.tcp.host"] = "127.0.0.1"
-            i2pOptions["i2cp.tcp.port"] = "7654"
-        }
         
         // options like tunnel length and quantity
 		I2PSession i2pSession
@@ -119,6 +162,7 @@ public class Core {
 		}
 		socketManager.getDefaultOptions().setReadTimeout(60000)
 		socketManager.getDefaultOptions().setConnectTimeout(30000)
+        socketManager.addDisconnectListener({eventBus.publish(new RouterDisconnectedEvent())} as DisconnectListener)
 		i2pSession = socketManager.getSession()
 		
         def destination = new Destination()
@@ -162,6 +206,11 @@ public class Core {
 		eventBus.register(FileDownloadedEvent.class, fileManager)
 		eventBus.register(FileUnsharedEvent.class, fileManager)
 		eventBus.register(SearchEvent.class, fileManager)
+        eventBus.register(DirectoryUnsharedEvent.class, fileManager)
+        
+        log.info("initializing mesh manager")
+        MeshManager meshManager = new MeshManager(fileManager, home, props)
+        eventBus.register(SourceDiscoveredEvent.class, meshManager)
 		
 		log.info "initializing persistence service"
 		persisterService = new PersisterService(new File(home, "files.json"), eventBus, 15000, fileManager)
@@ -186,7 +235,9 @@ public class Core {
 		cacheClient = new CacheClient(eventBus,hostCache, connectionManager, i2pSession, props, 10000)
         
         log.info("initializing update client")
-        updateClient = new UpdateClient(eventBus, i2pSession, myVersion, props)
+        updateClient = new UpdateClient(eventBus, i2pSession, myVersion, props, fileManager, me)
+        eventBus.register(FileDownloadedEvent.class, updateClient)
+        eventBus.register(UIResultBatchEvent.class, updateClient)
         
 		log.info("initializing connector")
 		I2PConnector i2pConnector = new I2PConnector(socketManager)
@@ -200,14 +251,17 @@ public class Core {
 		eventBus.register(ResultsEvent.class, searchManager)
 		
         log.info("initializing download manager")
-        downloadManager = new DownloadManager(eventBus, i2pConnector, home, me)
+        downloadManager = new DownloadManager(eventBus, trustService, meshManager, props, i2pConnector, home, me)
         eventBus.register(UIDownloadEvent.class, downloadManager)
         eventBus.register(UILoadedEvent.class, downloadManager)
         eventBus.register(FileDownloadedEvent.class, downloadManager)
         eventBus.register(UIDownloadCancelledEvent.class, downloadManager)
-        
+        eventBus.register(SourceDiscoveredEvent.class, downloadManager)
+        eventBus.register(UIDownloadPausedEvent.class, downloadManager)
+        eventBus.register(UIDownloadResumedEvent.class, downloadManager)
+                
         log.info("initializing upload manager")
-        UploadManager uploadManager = new UploadManager(eventBus, fileManager)
+        uploadManager = new UploadManager(eventBus, fileManager, meshManager, downloadManager)
         
         log.info("initializing connection establisher")
         connectionEstablisher = new ConnectionEstablisher(eventBus, i2pConnector, props, connectionManager, hostCache)
@@ -220,6 +274,8 @@ public class Core {
         log.info("initializing directory watcher")
         directoryWatcher = new DirectoryWatcher(eventBus, fileManager)
         eventBus.register(FileSharedEvent.class, directoryWatcher)
+        eventBus.register(AllFilesLoadedEvent.class, directoryWatcher)
+        eventBus.register(DirectoryUnsharedEvent.class, directoryWatcher)
         
         log.info("initializing hasher service")
         hasherService = new HasherService(new FileHasher(), eventBus, fileManager)
@@ -228,7 +284,6 @@ public class Core {
     
     public void startServices() {
         hasherService.start()
-        directoryWatcher.start()
         trustService.start()
         trustService.waitForLoad()
         hostCache.start()
@@ -241,6 +296,10 @@ public class Core {
     }
     
     public void shutdown() {
+        if (!shutdown.compareAndSet(false, true)) {
+            log.info("already shutting down")
+            return
+        }
         log.info("shutting down download manageer")
         downloadManager.shutdown()
         log.info("shutting down connection acceeptor")
@@ -251,8 +310,25 @@ public class Core {
         directoryWatcher.stop()
         log.info("shutting down connection manager")
         connectionManager.shutdown()
+        if (router != null) {
+            log.info("shutting down embedded router")
+            router.shutdown(0)
+        }
     }
 
+    static class RouterContextMetaClass extends DelegatingMetaClass {
+        private final Object logManager = new MuWireLogManager()
+        RouterContextMetaClass() {
+            super(RouterContext.class)
+        }
+        
+        Object invokeMethod(Object object, String name, Object[] args) {
+            if (name == "logManager")
+                return logManager
+            super.invokeMethod(object, name, args)
+        }
+    }
+    
     static main(args) {
         def home = System.getProperty("user.home") + File.separator + ".MuWire"
         home = new File(home)
@@ -277,7 +353,7 @@ public class Core {
             }
         }
         
-        Core core = new Core(props, home, "0.2.9")
+        Core core = new Core(props, home, "0.4.5")
         core.startServices()
         
         // ... at the end, sleep or execute script

core/src/main/groovy/com/muwire/core/MuWireSettings.groovy

@@ -13,11 +13,18 @@ class MuWireSettings {
     boolean allowUntrusted
     int downloadRetryInterval
     int updateCheckInterval
+    boolean autoDownloadUpdate
+    String updateType
     String nickname
     File downloadLocation
     CrawlerResponse crawlerResponse
     boolean shareDownloadedFiles
     Set<String> watchedDirectories
+    float downloadSequentialRatio
+    int hostClearInterval
+    int meshExpiration
+    boolean embeddedRouter
+    int inBw, outBw
     
 	MuWireSettings() {
         this(new Properties())
@@ -30,9 +37,17 @@ class MuWireSettings {
         nickname = props.getProperty("nickname","MuWireUser")
         downloadLocation = new File((String)props.getProperty("downloadLocation", 
             System.getProperty("user.home")))
-        downloadRetryInterval = Integer.parseInt(props.getProperty("downloadRetryInterval","5"))
-        updateCheckInterval = Integer.parseInt(props.getProperty("updateCheckInterval","36"))
+        downloadRetryInterval = Integer.parseInt(props.getProperty("downloadRetryInterval","1"))
+        updateCheckInterval = Integer.parseInt(props.getProperty("updateCheckInterval","24"))
+        autoDownloadUpdate = Boolean.parseBoolean(props.getProperty("autoDownloadUpdate","true"))
+        updateType = props.getProperty("updateType","jar")
         shareDownloadedFiles = Boolean.parseBoolean(props.getProperty("shareDownloadedFiles","true"))
+        downloadSequentialRatio = Float.valueOf(props.getProperty("downloadSequentialRatio","0.8"))
+        hostClearInterval = Integer.valueOf(props.getProperty("hostClearInterval","60"))
+        meshExpiration = Integer.valueOf(props.getProperty("meshExpiration","60"))
+        embeddedRouter = Boolean.valueOf(props.getProperty("embeddedRouter","false"))
+        inBw = Integer.valueOf(props.getProperty("inBw","256"))
+        outBw = Integer.valueOf(props.getProperty("outBw","128"))
         
         watchedDirectories = new HashSet<>()
         if (props.containsKey("watchedDirectories")) {
@@ -51,7 +66,15 @@ class MuWireSettings {
         props.setProperty("downloadLocation", downloadLocation.getAbsolutePath())
         props.setProperty("downloadRetryInterval", String.valueOf(downloadRetryInterval))
         props.setProperty("updateCheckInterval", String.valueOf(updateCheckInterval))
+        props.setProperty("autoDownloadUpdate", String.valueOf(autoDownloadUpdate))
+        props.setProperty("updateType",String.valueOf(updateType))
         props.setProperty("shareDownloadedFiles", String.valueOf(shareDownloadedFiles))
+        props.setProperty("downloadSequentialRatio", String.valueOf(downloadSequentialRatio))
+        props.setProperty("hostClearInterval", String.valueOf(hostClearInterval))
+        props.setProperty("meshExpiration", String.valueOf(meshExpiration))
+        props.setProperty("embeddedRouter", String.valueOf(embeddedRouter))
+        props.setProperty("inBw", String.valueOf(inBw))
+        props.setProperty("outBw", String.valueOf(outBw))
         
         if (!watchedDirectories.isEmpty()) {
             String encoded = watchedDirectories.stream().

core/src/main/groovy/com/muwire/core/Persona.groovy

@@ -82,4 +82,13 @@ public class Persona {
         Persona other = (Persona)o
         name.equals(other.name) && destination.equals(other.destination)
     }
+    
+    public static void main(String []args) {
+        if (args.length != 1) {
+            println "This utility decodes a bas64-encoded persona"
+            System.exit(1)
+        }
+        Persona p = new Persona(new ByteArrayInputStream(Base64.decode(args[0])))
+        println p.getHumanReadableName()
+    }
 }

core/src/main/groovy/com/muwire/core/RouterDisconnectedEvent.groovy

@@ -0,0 +1,4 @@
+package com.muwire.core
+
+class RouterDisconnectedEvent extends Event {
+}

core/src/main/groovy/com/muwire/core/connection/Connection.groovy

@@ -21,6 +21,9 @@ import net.i2p.data.Destination
 
 @Log
 abstract class Connection implements Closeable {
+    
+    private static final int SEARCHES = 10
+    private static final long INTERVAL = 1000
 
 	final EventBus eventBus
 	final Endpoint endpoint
@@ -32,6 +35,7 @@ abstract class Connection implements Closeable {
 	private final AtomicBoolean running = new AtomicBoolean()
 	private final BlockingQueue messages = new LinkedBlockingQueue()
 	private final Thread reader, writer
+    private final LinkedList<Long> searchTimestamps = new LinkedList<>()
 	
 	protected final String name
 	
@@ -156,7 +160,25 @@ abstract class Connection implements Closeable {
 		}
 	}
     
+    private boolean throttleSearch() {
+        final long now = System.currentTimeMillis()
+        if (searchTimestamps.size() < SEARCHES) {
+            searchTimestamps.addLast(now)
+            return false
+        }
+        Long oldest = searchTimestamps.getFirst()
+        if (now - oldest.longValue() < INTERVAL)
+            return true
+        searchTimestamps.addLast(now)
+        searchTimestamps.removeFirst()
+        false
+    }
+    
     protected void handleSearch(def search) {
+        if (throttleSearch()) {
+            log.info("dropping excessive search")
+            return
+        } 
         UUID uuid = UUID.fromString(search.uuid)
         byte [] infohash = null
         if (search.infohash != null) {

core/src/main/groovy/com/muwire/core/download/DownloadManager.groovy

@@ -3,6 +3,10 @@ package com.muwire.core.download
 import com.muwire.core.connection.I2PConnector
 import com.muwire.core.files.FileDownloadedEvent
 import com.muwire.core.files.FileHasher
+import com.muwire.core.mesh.Mesh
+import com.muwire.core.mesh.MeshManager
+import com.muwire.core.trust.TrustLevel
+import com.muwire.core.trust.TrustService
 import com.muwire.core.util.DataUtil
 
 import groovy.json.JsonBuilder
@@ -14,24 +18,33 @@ import net.i2p.util.ConcurrentHashSet
 
 import com.muwire.core.EventBus
 import com.muwire.core.InfoHash
+import com.muwire.core.MuWireSettings
 import com.muwire.core.Persona
 import com.muwire.core.UILoadedEvent
 
+import java.util.concurrent.ConcurrentHashMap
 import java.util.concurrent.Executor
 import java.util.concurrent.Executors
 
 public class DownloadManager {
     
     private final EventBus eventBus
+    private final TrustService trustService
+    private final MeshManager meshManager
+    private final MuWireSettings muSettings
     private final I2PConnector connector
     private final Executor executor
     private final File incompletes, home
     private final Persona me
     
-    private final Set<Downloader> downloaders = new ConcurrentHashSet<>()
+    private final Map<InfoHash, Downloader> downloaders = new ConcurrentHashMap<>()
     
-    public DownloadManager(EventBus eventBus, I2PConnector connector, File home, Persona me) {
+    public DownloadManager(EventBus eventBus, TrustService trustService, MeshManager meshManager, MuWireSettings muSettings,
+        I2PConnector connector, File home, Persona me) {
         this.eventBus = eventBus
+        this.trustService = trustService
+        this.meshManager = meshManager
+        this.muSettings = muSettings
         this.connector = connector
         this.incompletes = new File(home,"incompletes")
         this.home = home
@@ -61,17 +74,27 @@ public class DownloadManager {
         destinations.addAll(e.sources)
         destinations.remove(me.destination)
         
+        Pieces pieces = getPieces(infohash, size, pieceSize)
+        
         def downloader = new Downloader(eventBus, this, me, e.target, size,
             infohash, pieceSize, connector, destinations,
-            incompletes)
-        downloaders.add(downloader)
+            incompletes, pieces)
+        downloaders.put(infohash, downloader)
         persistDownloaders()
         executor.execute({downloader.download()} as Runnable)
         eventBus.publish(new DownloadStartedEvent(downloader : downloader))
     }
     
     public void onUIDownloadCancelledEvent(UIDownloadCancelledEvent e) {
-        downloaders.remove(e.downloader)
+        downloaders.remove(e.downloader.infoHash)
+        persistDownloaders()
+    }
+    
+    public void onUIDownloadPausedEvent(UIDownloadPausedEvent e) {
+        persistDownloaders()
+    }
+    
+    public void onUIDownloadResumedEvent(UIDownloadResumedEvent e) {
         persistDownloaders()
     }
     
@@ -99,23 +122,54 @@ public class DownloadManager {
                 byte [] root = Base64.decode(json.hashRoot)
                 infoHash = new InfoHash(root)
             }
+            
+            Pieces pieces = getPieces(infoHash, (long)json.length, json.pieceSizePow2)
+            
             def downloader = new Downloader(eventBus, this, me, file, (long)json.length,
-                infoHash, json.pieceSizePow2, connector, destinations, incompletes)
-            downloaders.add(downloader)
-            downloader.download()
+                infoHash, json.pieceSizePow2, connector, destinations, incompletes, pieces)
+            if (json.paused != null)
+                downloader.paused = json.paused
+            downloaders.put(infoHash, downloader)
+            downloader.readPieces()
+            if (!downloader.paused)
+                downloader.download()
             eventBus.publish(new DownloadStartedEvent(downloader : downloader))
         }
     }
     
+    private Pieces getPieces(InfoHash infoHash, long length, int pieceSizePow2) {
+        int pieceSize = 0x1 << pieceSizePow2
+        int nPieces = (int)(length / pieceSize)
+        if (length % pieceSize != 0)
+            nPieces++
+        Mesh mesh = meshManager.getOrCreate(infoHash, nPieces)
+        mesh.pieces
+    }
+    
+    void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
+        Downloader downloader = downloaders.get(e.infoHash)
+        if (downloader == null)
+            return
+        boolean ok = false
+        switch(trustService.getLevel(e.source.destination)) {
+            case TrustLevel.TRUSTED: ok = true; break
+            case TrustLevel.NEUTRAL: ok = muSettings.allowUntrusted; break
+            case TrustLevel.DISTRUSTED: ok = false; break
+        }
+        
+        if (ok)
+            downloader.addSource(e.source.destination)
+    }
+    
     void onFileDownloadedEvent(FileDownloadedEvent e) {
-        downloaders.remove(e.downloader)
+        downloaders.remove(e.downloader.infoHash)
         persistDownloaders()
     }
     
     private void persistDownloaders() {
         File downloadsFile = new File(home,"downloads.json")
         downloadsFile.withPrintWriter { writer -> 
-            downloaders.each { downloader ->
+            downloaders.values().each { downloader ->
                 if (!downloader.cancelled) {
                     def json = [:]
                     json.file = Base64.encode(DataUtil.encodei18nString(downloader.file.getAbsolutePath()))
@@ -132,6 +186,8 @@ public class DownloadManager {
                         json.hashList = Base64.encode(infoHash.hashList)
                     else
                         json.hashRoot = Base64.encode(infoHash.getRoot())
+                        
+                    json.paused = downloader.paused
                     writer.println(JsonOutput.toJson(json))
                 }
             }
@@ -139,7 +195,7 @@ public class DownloadManager {
     }
     
     public void shutdown() {
-        downloaders.each { it.stop() }
+        downloaders.values().each { it.stop() }
         Downloader.executorService.shutdownNow()
     }
 }

core/src/main/groovy/com/muwire/core/download/DownloadSession.groovy

@@ -3,8 +3,12 @@ package com.muwire.core.download;
 import net.i2p.data.Base64
 
 import com.muwire.core.Constants
+import com.muwire.core.EventBus
 import com.muwire.core.InfoHash
+import com.muwire.core.Persona
 import com.muwire.core.connection.Endpoint
+import com.muwire.core.util.DataUtil
+
 import static com.muwire.core.util.DataUtil.readTillRN
 
 import groovy.util.logging.Log
@@ -21,8 +25,7 @@ import java.util.logging.Level
 @Log
 class DownloadSession {
     
-    private static int SAMPLES = 10
-    
+    private final EventBus eventBus
     private final String meB64
     private final Pieces pieces
     private final InfoHash infoHash
@@ -30,15 +33,17 @@ class DownloadSession {
     private final File file
     private final int pieceSize
     private final long fileLength
+    private final Set<Integer> available
     private final MessageDigest digest
-    
-    private final LinkedList<Long> timestamps = new LinkedList<>()
-    private final LinkedList<Integer> reads = new LinkedList<>()
+
+    private long lastSpeedRead = System.currentTimeMillis()
+    private long dataSinceLastRead    
     
     private ByteBuffer mapped
     
-    DownloadSession(String meB64, Pieces pieces, InfoHash infoHash, Endpoint endpoint, File file, 
-        int pieceSize, long fileLength) {
+    DownloadSession(EventBus eventBus, String meB64, Pieces pieces, InfoHash infoHash, Endpoint endpoint, File file, 
+        int pieceSize, long fileLength, Set<Integer> available) {
+        this.eventBus = eventBus
         this.meB64 = meB64
         this.pieces = pieces
         this.endpoint = endpoint
@@ -46,6 +51,7 @@ class DownloadSession {
         this.file = file
         this.pieceSize = pieceSize
         this.fileLength = fileLength
+        this.available = available
         try {
             digest = MessageDigest.getInstance("SHA-256")
         } catch (NoSuchAlgorithmException impossible) {
@@ -63,7 +69,11 @@ class DownloadSession {
         OutputStream os = endpoint.getOutputStream()
         InputStream is = endpoint.getInputStream()
         
-        int piece = pieces.claim()
+        int piece
+        if (available.isEmpty())
+            piece = pieces.claim()
+        else
+            piece = pieces.claim(new HashSet<>(available))
         if (piece == -1)
             return false
         boolean unclaim = true
@@ -79,45 +89,79 @@ class DownloadSession {
         try {
             os.write("GET $root\r\n".getBytes(StandardCharsets.US_ASCII))
             os.write("Range: $start-$end\r\n".getBytes(StandardCharsets.US_ASCII))
-            os.write("X-Persona: $meB64\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
+            os.write("X-Persona: $meB64\r\n".getBytes(StandardCharsets.US_ASCII))
+            String xHave = DataUtil.encodeXHave(pieces.getDownloaded(), pieces.nPieces)
+            os.write("X-Have: $xHave\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) 
             os.flush()
-            String code = readTillRN(is)
-            if (code.startsWith("404 ")) {
+            String codeString = readTillRN(is)
+            int space = codeString.indexOf(' ')
+            if (space > 0)
+                codeString = codeString.substring(0, space)
+            
+            int code = Integer.parseInt(codeString.trim())
+            
+            if (code == 404) {
                 log.warning("file not found")
                 endpoint.close()
                 return false
             }
                 
-            if (code.startsWith("416 ")) {
-                log.warning("range $start-$end cannot be satisfied")
-                return // leave endpoint open
-            }
-            
-            if (!code.startsWith("200 ")) {
+            if (!(code == 200 || code == 416)) {
                 log.warning("unknown code $code")
                 endpoint.close()
                 return false
             }
 
             // parse all headers
-            Set<String> headers = new HashSet<>()
+            Map<String,String> headers = new HashMap<>()
             String header
-            while((header = readTillRN(is)) != "" && headers.size() < Constants.MAX_HEADERS) 
-                headers.add(header)
-
-            long receivedStart = -1
-            long receivedEnd = -1
-            for (String receivedHeader : headers) {
-                def group = (receivedHeader =~ /^Content-Range: (\d+)-(\d+)$/)
-                if (group.size() != 1) {
-                    log.info("ignoring header $receivedHeader")
-                    continue
-                } 
-
-                receivedStart = Long.parseLong(group[0][1])
-                receivedEnd = Long.parseLong(group[0][2])
+            while((header = readTillRN(is)) != "" && headers.size() < Constants.MAX_HEADERS) {
+                int colon = header.indexOf(':')
+                if (colon == -1 || colon == header.length() - 1)
+                    throw new IOException("invalid header $header")
+                String key = header.substring(0, colon)
+                String value = header.substring(colon + 1) 
+                headers[key] = value.trim()
             }
             
+            // prase X-Alt if present
+            if (headers.containsKey("X-Alt")) {
+                headers["X-Alt"].split(",").each {
+                    if (it.length() > 0) {
+                        byte [] raw = Base64.decode(it)
+                        Persona source = new Persona(new ByteArrayInputStream(raw))
+                        eventBus.publish(new SourceDiscoveredEvent(infoHash : infoHash, source : source))
+                    }
+                }
+            }
+
+            // parse X-Have if present
+            if (headers.containsKey("X-Have")) { 
+                DataUtil.decodeXHave(headers["X-Have"]).each {
+                    available.add(it)
+                }
+                if (!available.contains(piece))
+                    return true // try again next time
+            } else {
+                if (code != 200)
+                    throw new IOException("Code $code but no X-Have")
+                available.clear()
+            }
+            
+            if (code != 200)
+                return true
+            
+            String range = headers["Content-Range"]
+            if (range == null) 
+                throw new IOException("Code 200 but no Content-Range")
+            
+            def group = (range =~ /^(\d+)-(\d+)$/)
+            if (group.size() != 1) 
+                throw new IOException("invalid Content-Range header $range")
+
+            long receivedStart = Long.parseLong(group[0][1])
+            long receivedEnd = Long.parseLong(group[0][2])
+
             if (receivedStart != start || receivedEnd != end) {
                 log.warning("We don't support mismatching ranges yet")
                 endpoint.close()
@@ -140,18 +184,13 @@ class DownloadSession {
                         throw new IOException()
                     synchronized(this) {
                         mapped.put(tmp, 0, read)
-
-                        if (timestamps.size() == SAMPLES) {
-                            timestamps.removeFirst()
-                            reads.removeFirst()
-                        }
-                        timestamps.addLast(System.currentTimeMillis())
-                        reads.addLast(read)
+                        dataSinceLastRead += read
                     }
                 }
 
                 mapped.clear()
                 digest.update(mapped)
+                DataUtil.tryUnmap(mapped)
                 byte [] hash = digest.digest()
                 byte [] expected = new byte[32]
                 System.arraycopy(infoHash.getHashList(), piece * 32, expected, 0, 32)
@@ -176,24 +215,11 @@ class DownloadSession {
     }
     
     synchronized int speed() {
-        if (timestamps.size() < SAMPLES)
-            return 0
-        int totalRead = 0
-        int idx = 0
         final long now = System.currentTimeMillis()
-        
-        while(idx < SAMPLES && timestamps.get(idx) < now - 1000)
-            idx++
-        if (idx == SAMPLES)
-            return 0
-        if (idx == SAMPLES - 1)
-            return reads[idx]
-            
-        long interval = timestamps.last - timestamps[idx]
-        if (interval == 0) 
-            interval = 1
-        for (int i = idx; i < SAMPLES; i++)
-            totalRead += reads[idx]
-        (int)(totalRead * 1000.0 / interval)
+        long interval = Math.max(1000, now - lastSpeedRead)
+        lastSpeedRead = now;
+        int rv = (int) (dataSinceLastRead * 1000.0 / interval)
+        dataSinceLastRead = 0
+        rv
     }
 }

core/src/main/groovy/com/muwire/core/download/Downloader.groovy

@@ -7,6 +7,7 @@ import com.muwire.core.connection.Endpoint
 import java.nio.file.AtomicMoveNotSupportedException
 import java.nio.file.Files
 import java.nio.file.StandardCopyOption
+import java.time.Instant
 import java.util.concurrent.ConcurrentHashMap
 import java.util.concurrent.ExecutorService
 import java.util.concurrent.Executors
@@ -18,6 +19,7 @@ import com.muwire.core.DownloadedFile
 import com.muwire.core.EventBus
 import com.muwire.core.connection.I2PConnector
 import com.muwire.core.files.FileDownloadedEvent
+import com.muwire.core.util.DataUtil
 
 import groovy.util.logging.Log
 import net.i2p.data.Destination
@@ -25,7 +27,7 @@ import net.i2p.util.ConcurrentHashSet
 
 @Log
 public class Downloader {
-    public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, CANCELLED, FINISHED }
+    public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, CANCELLED, PAUSED, FINISHED }
     private enum WorkerState { CONNECTING, HASHLIST, DOWNLOADING, FINISHED}
     
     private static final ExecutorService executorService = Executors.newCachedThreadPool({r ->
@@ -53,14 +55,19 @@ public class Downloader {
     private final Set<Destination> successfulDestinations = new ConcurrentHashSet<>()
     
     
-    private volatile boolean cancelled
+    private volatile boolean cancelled, paused
     private final AtomicBoolean eventFired = new AtomicBoolean()
     private boolean piecesFileClosed
 
+    private ArrayList speedArr = new ArrayList<Integer>()
+    private int speedPos = 0
+    private int speedAvg = 0
+    private long timestamp = Instant.now().toEpochMilli()
+
     public Downloader(EventBus eventBus, DownloadManager downloadManager, 
         Persona me, File file, long length, InfoHash infoHash, 
         int pieceSizePow2, I2PConnector connector, Set<Destination> destinations,
-        File incompletes) {
+        File incompletes, Pieces pieces) {
         this.eventBus = eventBus
         this.me = me
         this.downloadManager = downloadManager
@@ -73,15 +80,12 @@ public class Downloader {
         this.incompleteFile = new File(incompletes, file.getName()+".part")
         this.pieceSizePow2 = pieceSizePow2
         this.pieceSize = 1 << pieceSizePow2
-        
-        int nPieces
-        if (length % pieceSize == 0)
-            nPieces = length / pieceSize
-        else
-            nPieces = length / pieceSize + 1
-        this.nPieces = nPieces
-        
-        pieces = new Pieces(nPieces, Constants.DOWNLOAD_SEQUENTIAL_RATIO)
+        this.pieces = pieces
+        this.nPieces = pieces.nPieces
+
+        // default size suitable for an average of 5 seconds / 5 elements / 5 interval units
+        // it's easily adjustable by resizing the size of speedArr
+        this.speedArr = [ 0, 0, 0, 0, 0 ]
     }
     
     public synchronized InfoHash getInfoHash() {
@@ -130,19 +134,43 @@ public class Downloader {
     
     
     public int speed() {
-        int total = 0
+        int currSpeed = 0
         if (getCurrentState() == DownloadState.DOWNLOADING) {
             activeWorkers.values().each {
                 if (it.currentState == WorkerState.DOWNLOADING)
-                    total += it.speed()
+                    currSpeed += it.speed()
             }
         }
-        total
+
+        // normalize to speedArr.size
+        currSpeed /= speedArr.size()
+
+        // compute new speedAvg and update speedArr
+        if ( speedArr[speedPos] > speedAvg ) {
+            speedAvg = 0
+        } else {
+            speedAvg -= speedArr[speedPos]
+        }
+        speedAvg += currSpeed
+        speedArr[speedPos] = currSpeed
+        // this might be necessary due to rounding errors
+        if (speedAvg < 0)
+            speedAvg = 0
+
+        // rolling index over the speedArr
+        speedPos++
+        if (speedPos >= speedArr.size())
+            speedPos=0
+
+        speedAvg
     }
     
     public DownloadState getCurrentState() {
         if (cancelled)
             return DownloadState.CANCELLED
+        if (paused)
+            return DownloadState.PAUSED
+            
         boolean allFinished = true
         activeWorkers.values().each { 
             allFinished &= it.currentState == WorkerState.FINISHED
@@ -187,6 +215,12 @@ public class Downloader {
             piecesFile.delete()
         }
         incompleteFile.delete()
+        pieces.clearAll()
+    }
+    
+    public void pause() {
+        paused = true
+        stop()
     }
     
     void stop() {
@@ -205,6 +239,8 @@ public class Downloader {
     }
     
     public void resume() {
+        paused = false
+        readPieces()
         destinations.each { destination ->
             def worker = activeWorkers.get(destination)
             if (worker != null) {
@@ -221,12 +257,21 @@ public class Downloader {
         }
     }
     
+    void addSource(Destination d) {
+        if (activeWorkers.containsKey(d))
+            return
+        DownloadWorker newWorker = new DownloadWorker(d)
+        activeWorkers.put(d, newWorker)
+        executorService.submit(newWorker)
+    }
+    
     class DownloadWorker implements Runnable {
         private final Destination destination
         private volatile WorkerState currentState
         private volatile Thread downloadThread
         private Endpoint endpoint
         private volatile DownloadSession currentSession
+        private final Set<Integer> available = new HashSet<>()
                 
         DownloadWorker(Destination destination) {
             this.destination = destination
@@ -247,7 +292,8 @@ public class Downloader {
                 currentState = WorkerState.DOWNLOADING
                 boolean requestPerformed
                 while(!pieces.isComplete()) {
-                    currentSession = new DownloadSession(me.toBase64(), pieces, getInfoHash(), endpoint, incompleteFile, pieceSize, length)
+                    currentSession = new DownloadSession(eventBus, me.toBase64(), pieces, getInfoHash(), 
+                        endpoint, incompleteFile, pieceSize, length, available)
                     requestPerformed = currentSession.request()
                     if (!requestPerformed)
                         break
@@ -255,7 +301,7 @@ public class Downloader {
                     writePieces()
                 }
             } catch (Exception bad) {
-                log.log(Level.WARNING,"Exception while downloading",bad)
+                log.log(Level.WARNING,"Exception while downloading",DataUtil.findRoot(bad))
             } finally {
                 currentState = WorkerState.FINISHED
                 if (pieces.isComplete() && eventFired.compareAndSet(false, true)) {

core/src/main/groovy/com/muwire/core/download/Pieces.groovy

@@ -38,6 +38,18 @@ class Pieces {
         }
     }
     
+    synchronized int claim(Set<Integer> available) {
+        for (int i = claimed.nextSetBit(0); i >= 0; i = claimed.nextSetBit(i+1))
+            available.remove(i)
+        if (available.isEmpty())
+            return -1
+        List<Integer> toList = available.toList()
+        Collections.shuffle(toList)
+        int rv = toList[0]
+        claimed.set(rv)
+        rv
+    }
+    
     synchronized def getDownloaded() {
         def rv = []
         for (int i = done.nextSetBit(0); i >= 0; i = done.nextSetBit(i+1)) {
@@ -62,4 +74,13 @@ class Pieces {
     synchronized int donePieces() {
         done.cardinality()
     }
+    
+    synchronized boolean isDownloaded(int piece) {
+        done.get(piece)
+    }
+    
+    synchronized void clearAll() {
+        done.clear()
+        claimed.clear()
+    }
 }

core/src/main/groovy/com/muwire/core/download/SourceDiscoveredEvent.groovy

@@ -0,0 +1,10 @@
+package com.muwire.core.download
+
+import com.muwire.core.Event
+import com.muwire.core.InfoHash
+import com.muwire.core.Persona
+
+class SourceDiscoveredEvent extends Event {
+    InfoHash infoHash
+    Persona source
+}

core/src/main/groovy/com/muwire/core/download/UIDownloadPausedEvent.groovy

@@ -0,0 +1,6 @@
+package com.muwire.core.download
+
+import com.muwire.core.Event
+
+class UIDownloadPausedEvent extends Event {
+}

core/src/main/groovy/com/muwire/core/download/UIDownloadResumedEvent.groovy

@@ -0,0 +1,6 @@
+package com.muwire.core.download
+
+import com.muwire.core.Event
+
+class UIDownloadResumedEvent extends Event {
+}

core/src/main/groovy/com/muwire/core/files/DirectoryUnsharedEvent.groovy

@@ -0,0 +1,7 @@
+package com.muwire.core.files
+
+import com.muwire.core.Event
+
+class DirectoryUnsharedEvent extends Event {
+    File directory
+}

core/src/main/groovy/com/muwire/core/files/DirectoryWatcher.groovy

@@ -35,6 +35,7 @@ class DirectoryWatcher {
     private final FileManager fileManager
     private final Thread watcherThread, publisherThread
     private final Map<File, Long> waitingFiles = new ConcurrentHashMap<>()
+    private final Map<File, WatchKey> watchedDirectories = new ConcurrentHashMap<>()
     private WatchService watchService
     private volatile boolean shutdown
     
@@ -47,7 +48,7 @@ class DirectoryWatcher {
         publisherThread.setDaemon(true)
     }
     
-    void start() {
+    void onAllFilesLoadedEvent(AllFilesLoadedEvent e) {
         watchService = FileSystems.getDefault().newWatchService()
         watcherThread.start()
         publisherThread.start()
@@ -55,18 +56,24 @@ class DirectoryWatcher {
     
     void stop() {
         shutdown = true
-        watcherThread.interrupt()
-        publisherThread.interrupt()
-        watchService.close()
+        watcherThread?.interrupt()
+        publisherThread?.interrupt()
+        watchService?.close()
     }
     
     void onFileSharedEvent(FileSharedEvent e) {
         if (!e.file.isDirectory())
             return
         Path path = e.file.getCanonicalFile().toPath()
-        path.register(watchService, kinds)
+        WatchKey wk = path.register(watchService, kinds)
+        watchedDirectories.put(e.file, wk)
         
     }
+    
+    void onDirectoryUnsharedEvent(DirectoryUnsharedEvent e) {
+        WatchKey wk = watchedDirectories.remove(e.directory)
+        wk?.cancel()
+    }
 
     private void watch() {
         try {
@@ -113,7 +120,7 @@ class DirectoryWatcher {
     
     private static File join(Path parent, Path path) {
         File parentFile = parent.toFile().getCanonicalFile()
-        new File(parentFile, path.toFile().getName())
+        new File(parentFile, path.toFile().getName()).getCanonicalFile()
     }
     
     private void publish() {

core/src/main/groovy/com/muwire/core/files/FileHashedEvent.groovy

@@ -7,4 +7,10 @@ class FileHashedEvent extends Event {
 
 	SharedFile sharedFile
 	String error
+    
+    @Override
+    public String toString() {
+        super.toString() + " sharedFile " + sharedFile?.file.getAbsolutePath() + " error: $error"
+    }
+    
 }

core/src/main/groovy/com/muwire/core/files/FileHasher.groovy

@@ -1,6 +1,7 @@
 package com.muwire.core.files
 
 import com.muwire.core.InfoHash
+import com.muwire.core.util.DataUtil
 
 import net.i2p.data.Base64
 
@@ -18,6 +19,8 @@ class FileHasher {
 	/**
 	 * @param size of the file to be shared
 	 * @return the size of each piece in power of 2
+     * piece size is minimum 128 KBytees and maximum 16 MBytes in power of 2 steps (2^17 - 2^24)
+     * there can be up to 8192 pieces maximum per file
 	 */
 	static int getPieceSize(long size) {
 		if (size <= 0x1 << 30)
@@ -57,6 +60,7 @@ class FileHasher {
 			for (int i = 0; i < numPieces - 1; i++) {
 				buf = raf.getChannel().map(MapMode.READ_ONLY, ((long)size) * i, size)
 				digest.update buf
+                DataUtil.tryUnmap(buf)
 				output.write(digest.digest(), 0, 32)
 			}
 			def lastPieceLength = length - (numPieces - 1) * ((long)size)

core/src/main/groovy/com/muwire/core/files/FileManager.groovy

@@ -135,4 +135,16 @@ class FileManager {
         }
         rv
     }
+    
+    void onDirectoryUnsharedEvent(DirectoryUnsharedEvent e) {
+        e.directory.listFiles().each { 
+            if (it.isDirectory())
+                eventBus.publish(new DirectoryUnsharedEvent(directory : it))
+            else {
+                SharedFile sf = fileToSharedFile.get(it)
+                if (sf != null)
+                    eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
+            }
+        }
+    }
 }

core/src/main/groovy/com/muwire/core/files/FileSharedEvent.groovy

@@ -5,4 +5,9 @@ import com.muwire.core.Event
 class FileSharedEvent extends Event {
 
 	File file
+    
+    @Override
+    public String toString() {
+        return super.toString() + " file: "+file.getAbsolutePath()
+    }
 }

core/src/main/groovy/com/muwire/core/files/HasherService.groovy

@@ -24,7 +24,7 @@ class HasherService {
 	}
 	
 	void onFileSharedEvent(FileSharedEvent evt) {
-        if (fileManager.fileToSharedFile.containsKey(evt.file))
+        if (fileManager.fileToSharedFile.containsKey(evt.file.getCanonicalFile()))
             return
 		executor.execute( { -> process(evt.file) } as Runnable)
 	}

core/src/main/groovy/com/muwire/core/hostcache/CacheServers.groovy

@@ -6,7 +6,8 @@ class CacheServers {
 
 	private static final int TO_GIVE = 3
 	private static Set<Destination> CACHES = [
-			new Destination("Wddh2E6FyyXBF7SvUYHKdN-vjf3~N6uqQWNeBDTM0P33YjiQCOsyedrjmDZmWFrXUJfJLWnCb5bnKezfk4uDaMyj~uvDG~yvLVcFgcPWSUd7BfGgym-zqcG1q1DcM8vfun-US7YamBlmtC6MZ2j-~Igqzmgshita8aLPCfNAA6S6e2UMjjtG7QIXlxpMec75dkHdJlVWbzrk9z8Qgru3YIk0UztYgEwDNBbm9wInsbHhr3HtAfa02QcgRVqRN2PnQXuqUJs7R7~09FZPEviiIcUpkY3FeyLlX1sgQFBeGeA96blaPvZNGd6KnNdgfLgMebx5SSxC-N4KZMSMBz5cgonQF3~m2HHFRSI85zqZNG5X9bJN85t80ltiv1W1es8ZnQW4es11r7MrvJNXz5bmSH641yJIvS6qI8OJJNpFVBIQSXLD-96TayrLQPaYw~uNZ-eXaE6G5dYhiuN8xHsFI1QkdaUaVZnvDGfsRbpS5GtpUbBDbyLkdPurG0i7dN1wAAAA")
+			new Destination("Wddh2E6FyyXBF7SvUYHKdN-vjf3~N6uqQWNeBDTM0P33YjiQCOsyedrjmDZmWFrXUJfJLWnCb5bnKezfk4uDaMyj~uvDG~yvLVcFgcPWSUd7BfGgym-zqcG1q1DcM8vfun-US7YamBlmtC6MZ2j-~Igqzmgshita8aLPCfNAA6S6e2UMjjtG7QIXlxpMec75dkHdJlVWbzrk9z8Qgru3YIk0UztYgEwDNBbm9wInsbHhr3HtAfa02QcgRVqRN2PnQXuqUJs7R7~09FZPEviiIcUpkY3FeyLlX1sgQFBeGeA96blaPvZNGd6KnNdgfLgMebx5SSxC-N4KZMSMBz5cgonQF3~m2HHFRSI85zqZNG5X9bJN85t80ltiv1W1es8ZnQW4es11r7MrvJNXz5bmSH641yJIvS6qI8OJJNpFVBIQSXLD-96TayrLQPaYw~uNZ-eXaE6G5dYhiuN8xHsFI1QkdaUaVZnvDGfsRbpS5GtpUbBDbyLkdPurG0i7dN1wAAAA"),
+            new Destination("JC63wJNOqSJmymkj4~UJWywBTvDGikKMoYP0HX2Wz9c5l3otXSkwnxWAFL4cKr~Ygh3BNNi2t93vuLIiI1W8AsE42kR~PwRx~Y-WvIHXR6KUejRmOp-n8WidtjKg9k4aDy428uSOedqXDxys5mpoeQXwDsv1CoPTTwnmb1GWFy~oTGIsCguCl~aJWGnqiKarPO3GJQ~ev-NbvAQzUfC3HeP1e6pdI5CGGjExahTCID5UjpJw8GaDXWlGmYWWH303Xu4x-vAHQy1dJLsOBCn8dZravsn5BKJk~j0POUon45CCx-~NYtaPe0Itt9cMdD2ciC76Rep1D0X0sm1SjlSs8sZ52KmF3oaLZ6OzgI9QLMIyBUrfi41sK5I0qTuUVBAkvW1xr~L-20dYJ9TrbOaOb2-vDIfKaxVi6xQOuhgQDiSBhd3qv2m0xGu-BM9DQYfNA0FdMjnZmqjmji9RMavzQSsVFIbQGLbrLepiEFlb7TseCK5UtRp8TxnG7L4gbYevBQAEAAcAAA==")
 		]
 
 	static List<Destination> getCacheServers() {

core/src/main/groovy/com/muwire/core/hostcache/Host.groovy

@@ -5,14 +5,15 @@ import net.i2p.data.Destination
 class Host {
 
 	private static final int MAX_FAILURES = 3
-    private static final int CLEAR_INTERVAL = 60 * 60 * 1000
 	
 	final Destination destination
+    private final int clearInterval
 	int failures,successes
     long lastAttempt
 	
-	public Host(Destination destination) {
+	public Host(Destination destination, int clearInterval) {
 		this.destination = destination
+        this.clearInterval = clearInterval
 	}
 
 	synchronized void onConnect() {
@@ -40,6 +41,6 @@ class Host {
     }
     
     synchronized void canTryAgain() {
-        System.currentTimeMillis() - lastAttempt > CLEAR_INTERVAL
+        System.currentTimeMillis() - lastAttempt > (clearInterval * 60 * 1000)
     }
 }

core/src/main/groovy/com/muwire/core/hostcache/HostCache.groovy

@@ -52,7 +52,7 @@ class HostCache extends Service {
             hosts.get(e.destination).clearFailures()
             return
 		}
-		Host host = new Host(e.destination)
+		Host host = new Host(e.destination, settings.hostClearInterval)
 		if (allowHost(host)) {
 			hosts.put(e.destination, host)
 		}
@@ -64,7 +64,7 @@ class HostCache extends Service {
 		Destination dest = e.endpoint.destination
 		Host host = hosts.get(dest)
 		if (host == null) {
-			host = new Host(dest)
+			host = new Host(dest, settings.hostClearInterval)
 			hosts.put(dest, host)
 		}
 
@@ -106,7 +106,7 @@ class HostCache extends Service {
 			storage.eachLine {
 				def entry = slurper.parseText(it)
 				Destination dest = new Destination(entry.destination)
-				Host host = new Host(dest)
+				Host host = new Host(dest, settings.hostClearInterval)
 				host.failures = Integer.valueOf(String.valueOf(entry.failures))
 				host.successes = Integer.valueOf(String.valueOf(entry.successes))
                 if (entry.lastAttempt != null)

core/src/main/groovy/com/muwire/core/mesh/Mesh.groovy

@@ -0,0 +1,28 @@
+package com.muwire.core.mesh
+
+import com.muwire.core.InfoHash
+import com.muwire.core.Persona
+import com.muwire.core.download.Pieces
+
+import net.i2p.data.Destination
+import net.i2p.util.ConcurrentHashSet
+
+class Mesh {
+    private final InfoHash infoHash
+    private final Set<Persona> sources = new ConcurrentHashSet<>()
+    private final Pieces pieces
+    
+    Mesh(InfoHash infoHash, Pieces pieces) {
+        this.infoHash = infoHash
+        this.pieces = pieces
+    }
+    
+    Set<Persona> getRandom(int n, Persona exclude) {
+        List<Persona> tmp = new ArrayList<>(sources)
+        tmp.remove(exclude)
+        Collections.shuffle(tmp)
+        if (tmp.size() < n)
+            return tmp
+        tmp[0..n-1]
+    }
+}

core/src/main/groovy/com/muwire/core/mesh/MeshManager.groovy

@@ -0,0 +1,102 @@
+package com.muwire.core.mesh
+
+import java.util.stream.Collectors
+
+import com.muwire.core.Constants
+import com.muwire.core.InfoHash
+import com.muwire.core.MuWireSettings
+import com.muwire.core.Persona
+import com.muwire.core.download.Pieces
+import com.muwire.core.download.SourceDiscoveredEvent
+import com.muwire.core.files.FileManager
+import com.muwire.core.util.DataUtil
+
+import groovy.json.JsonOutput
+import groovy.json.JsonSlurper
+import net.i2p.data.Base64
+
+class MeshManager {
+    
+    private final Map<InfoHash, Mesh> meshes = Collections.synchronizedMap(new HashMap<>())
+    private final FileManager fileManager
+    private final File home
+    private final MuWireSettings settings
+    
+    MeshManager(FileManager fileManager, File home, MuWireSettings settings) {
+        this.fileManager = fileManager
+        this.home = home
+        this.settings = settings
+        load()
+    }
+    
+    Mesh get(InfoHash infoHash) {
+        meshes.get(infoHash)
+    }
+    
+    Mesh getOrCreate(InfoHash infoHash, int nPieces) {
+        synchronized(meshes) {
+            if (meshes.containsKey(infoHash))
+                return meshes.get(infoHash)
+            Pieces pieces = new Pieces(nPieces, settings.downloadSequentialRatio)
+            if (fileManager.rootToFiles.containsKey(infoHash)) {
+                for (int i = 0; i < nPieces; i++)
+                    pieces.markDownloaded(i)
+            }
+            Mesh rv = new Mesh(infoHash, pieces)
+            meshes.put(infoHash, rv)
+            return rv
+        }
+    }
+    
+    void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
+        Mesh mesh = meshes.get(e.infoHash)
+        if (mesh == null)
+            return
+       mesh.sources.add(e.source)
+       save()
+    }
+    
+    private void save() {
+        File meshFile = new File(home, "mesh.json")
+        synchronized(meshes) {
+            meshFile.withPrintWriter { writer ->
+                meshes.values().each { mesh ->
+                    def json = [:]
+                    json.timestamp = System.currentTimeMillis()
+                    json.infoHash = Base64.encode(mesh.infoHash.getRoot())
+                    json.sources = mesh.sources.stream().map({it.toBase64()}).collect(Collectors.toList())
+                    json.nPieces = mesh.pieces.nPieces
+                    json.xHave = DataUtil.encodeXHave(mesh.pieces.downloaded, mesh.pieces.nPieces)
+                    writer.println(JsonOutput.toJson(json))
+                }
+            }
+        }
+    }
+    
+    private void load() {
+        File meshFile = new File(home, "mesh.json")
+        if (!meshFile.exists())
+            return
+        long now = System.currentTimeMillis()
+        JsonSlurper slurper = new JsonSlurper()
+        meshFile.eachLine { 
+            def json = slurper.parseText(it)
+            if (now - json.timestamp > settings.meshExpiration * 60 * 1000)
+                return
+            InfoHash infoHash = new InfoHash(Base64.decode(json.infoHash))
+            Pieces pieces = new Pieces(json.nPieces, settings.downloadSequentialRatio)
+            
+            Mesh mesh = new Mesh(infoHash, pieces)
+            json.sources.each { source -> 
+                Persona persona = new Persona(new ByteArrayInputStream(Base64.decode(source)))
+                mesh.sources.add(persona)
+            }
+            
+            if (json.xHave != null) 
+                DataUtil.decodeXHave(json.xHave).each { pieces.markDownloaded(it) }
+                
+            if (!mesh.sources.isEmpty())
+                meshes.put(infoHash, mesh)
+        }
+    }
+}

core/src/main/groovy/com/muwire/core/search/ResultsSender.groovy

@@ -11,6 +11,7 @@ import java.util.concurrent.Executor
 import java.util.concurrent.Executors
 import java.util.concurrent.ThreadFactory
 import java.util.concurrent.atomic.AtomicInteger
+import java.util.logging.Level
 import java.util.stream.Collectors
 
 import com.muwire.core.DownloadedFile
@@ -83,50 +84,54 @@ class ResultsSender {
         
         @Override
         public void run() {
-            byte [] tmp = new byte[InfoHash.SIZE]
-            JsonOutput jsonOutput = new JsonOutput()
-            Endpoint endpoint = null;
             try {
-                endpoint = connector.connect(target)
-                DataOutputStream os = new DataOutputStream(endpoint.getOutputStream())
-                os.write("POST $uuid\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
-                me.write(os)
-                os.writeShort((short)results.length)
-                results.each {
-                    byte [] name = it.getFile().getName().getBytes(StandardCharsets.UTF_8)
-                    def baos = new ByteArrayOutputStream()
-                    def daos = new DataOutputStream(baos)
-                    daos.writeShort((short) name.length)
-                    daos.write(name)
-                    daos.flush()
-                    String encodedName = Base64.encode(baos.toByteArray())
-                    def obj = [:]
-                    obj.type = "Result"
-                    obj.version = oobInfohash ? 2 : 1
-                    obj.name = encodedName
-                    obj.infohash = Base64.encode(it.getInfoHash().getRoot())
-                    obj.size = it.getFile().length()
-                    obj.pieceSize = it.getPieceSize()
-                    if (!oobInfohash) {
-                        byte [] hashList = it.getInfoHash().getHashList()
-                        def hashListB64 = []
-                        for (int i = 0; i < hashList.length / InfoHash.SIZE; i++) {
-                            System.arraycopy(hashList, InfoHash.SIZE * i, tmp, 0, InfoHash.SIZE)
-                            hashListB64 << Base64.encode(tmp)
+                byte [] tmp = new byte[InfoHash.SIZE]
+                JsonOutput jsonOutput = new JsonOutput()
+                Endpoint endpoint = null;
+                try {
+                    endpoint = connector.connect(target)
+                    DataOutputStream os = new DataOutputStream(endpoint.getOutputStream())
+                    os.write("POST $uuid\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
+                    me.write(os)
+                    os.writeShort((short)results.length)
+                    results.each {
+                        byte [] name = it.getFile().getName().getBytes(StandardCharsets.UTF_8)
+                        def baos = new ByteArrayOutputStream()
+                        def daos = new DataOutputStream(baos)
+                        daos.writeShort((short) name.length)
+                        daos.write(name)
+                        daos.flush()
+                        String encodedName = Base64.encode(baos.toByteArray())
+                        def obj = [:]
+                        obj.type = "Result"
+                        obj.version = oobInfohash ? 2 : 1
+                        obj.name = encodedName
+                        obj.infohash = Base64.encode(it.getInfoHash().getRoot())
+                        obj.size = it.getFile().length()
+                        obj.pieceSize = it.getPieceSize()
+                        if (!oobInfohash) {
+                            byte [] hashList = it.getInfoHash().getHashList()
+                            def hashListB64 = []
+                            for (int i = 0; i < hashList.length / InfoHash.SIZE; i++) {
+                                System.arraycopy(hashList, InfoHash.SIZE * i, tmp, 0, InfoHash.SIZE)
+                                hashListB64 << Base64.encode(tmp)
+                            }
+                            obj.hashList = hashListB64
                         }
-                        obj.hashList = hashListB64
+
+                        if (it instanceof DownloadedFile)
+                            obj.sources = it.sources.stream().map({dest -> dest.toBase64()}).collect(Collectors.toSet())
+
+                        def json = jsonOutput.toJson(obj)
+                        os.writeShort((short)json.length())
+                        os.write(json.getBytes(StandardCharsets.US_ASCII))
                     }
-                    
-                    if (it instanceof DownloadedFile)
-                        obj.sources = it.sources.stream().map({dest -> dest.toBase64()}).collect(Collectors.toSet())
-                    
-                    def json = jsonOutput.toJson(obj)
-                    os.writeShort((short)json.length())
-                    os.write(json.getBytes(StandardCharsets.US_ASCII))
+                    os.flush()
+                } finally {
+                    endpoint?.close()
                 }
-                os.flush()
-            } finally {
-                endpoint?.close()
+            } catch (Exception e) {
+                log.log(Level.WARNING, "problem sending results",e)
             }
         }
     }

core/src/main/groovy/com/muwire/core/search/SearchIndex.groovy

@@ -33,7 +33,10 @@ class SearchIndex {
 	
 	private static String[] split(String source) {
 		source = source.replaceAll(Constants.SPLIT_PATTERN, " ").toLowerCase()
-		source.split(" ")
+		String [] split = source.split(" ")
+        def rv = []
+        split.each { if (it.length() > 0) rv << it }
+        rv.toArray(new String[0])
 	}
 	
 	String[] search(List<String> terms) {

core/src/main/groovy/com/muwire/core/update/UpdateClient.groovy

@@ -3,7 +3,15 @@ package com.muwire.core.update
 import java.util.logging.Level
 
 import com.muwire.core.EventBus
+import com.muwire.core.InfoHash
 import com.muwire.core.MuWireSettings
+import com.muwire.core.Persona
+import com.muwire.core.download.UIDownloadEvent
+import com.muwire.core.files.FileDownloadedEvent
+import com.muwire.core.files.FileManager
+import com.muwire.core.search.QueryEvent
+import com.muwire.core.search.SearchEvent
+import com.muwire.core.search.UIResultBatchEvent
 
 import groovy.json.JsonOutput
 import groovy.json.JsonSlurper
@@ -13,6 +21,7 @@ import net.i2p.client.I2PSessionMuxedListener
 import net.i2p.client.SendMessageOptions
 import net.i2p.client.datagram.I2PDatagramDissector
 import net.i2p.client.datagram.I2PDatagramMaker
+import net.i2p.data.Base64
 import net.i2p.util.VersionComparator
 
 @Log
@@ -21,16 +30,24 @@ class UpdateClient {
     final I2PSession session
     final String myVersion
     final MuWireSettings settings
+    final FileManager fileManager
+    final Persona me
     
     private final Timer timer
     
     private long lastUpdateCheckTime
     
-    UpdateClient(EventBus eventBus, I2PSession session, String myVersion, MuWireSettings settings) {
+    private volatile InfoHash updateInfoHash
+    private volatile String version, signer
+    private volatile boolean updateDownloading
+    
+    UpdateClient(EventBus eventBus, I2PSession session, String myVersion, MuWireSettings settings, FileManager fileManager, Persona me) {
         this.eventBus = eventBus
         this.session = session
         this.myVersion = myVersion
         this.settings = settings
+        this.fileManager = fileManager
+        this.me = me
         timer = new Timer("update-client",true)
     }
     
@@ -43,6 +60,24 @@ class UpdateClient {
         timer.cancel()
     }
     
+    void onUIResultBatchEvent(UIResultBatchEvent results) {
+        if (results.results[0].infohash != updateInfoHash)
+            return
+        if (updateDownloading)
+            return
+        updateDownloading = true
+        def file = new File(settings.downloadLocation, results.results[0].name)
+        def downloadEvent = new UIDownloadEvent(result: results.results[0], sources : results.results[0].sources, target : file)
+        eventBus.publish(downloadEvent)
+    }
+ 
+    void onFileDownloadedEvent(FileDownloadedEvent e) {
+        if (e.downloadedFile.infoHash != updateInfoHash)
+            return
+        updateDownloading = false
+        eventBus.publish(new UpdateDownloadedEvent(version : version, signer : signer))
+    }
+    
     private void checkUpdate() {
         final long now = System.currentTimeMillis()
         if (lastUpdateCheckTime > 0) {
@@ -106,8 +141,32 @@ class UpdateClient {
                     return
                 }
                 
-                log.info("new version $payload.version available, publishing event")
-                eventBus.publish(new UpdateAvailableEvent(version : payload.version, signer : payload.signer, infoHash : payload.infoHash))
+                String infoHash
+                if (settings.updateType == "jar") {
+                    infoHash = payload.infoHash
+                } else
+                    infoHash = payload[settings.updateType]
+                
+              
+                if (!settings.autoDownloadUpdate) {
+                    log.info("new version $payload.version available, publishing event")
+                    eventBus.publish(new UpdateAvailableEvent(version : payload.version, signer : payload.signer, infoHash : infoHash))
+                } else {
+                    log.info("new version $payload.version available")
+                    updateInfoHash = new InfoHash(Base64.decode(infoHash))
+                    if (fileManager.rootToFiles.containsKey(updateInfoHash))
+                        eventBus.publish(new UpdateDownloadedEvent(version : payload.version, signer : payload.signer))
+                    else {
+                        updateDownloading = false
+                        version = payload.version
+                        signer = payload.signer
+                        log.info("starting search for new version hash $payload.infoHash")
+                        def searchEvent = new SearchEvent(searchHash : updateInfoHash.getRoot(), uuid : UUID.randomUUID(), oobInfohash : true)
+                        def queryEvent = new QueryEvent(searchEvent : searchEvent, firstHop : true, replyTo : me.destination,
+                            receivedOn : me.destination, originator : me)
+                        eventBus.publish(queryEvent)
+                    }
+                }
                 
             } catch (Exception e) {
                 log.log(Level.WARNING,"Invalid datagram",e)

core/src/main/groovy/com/muwire/core/update/UpdateDownloadedEvent.groovy

@@ -0,0 +1,8 @@
+package com.muwire.core.update
+
+import com.muwire.core.Event
+
+class UpdateDownloadedEvent extends Event {
+    String version
+    String signer
+}

core/src/main/groovy/com/muwire/core/upload/ContentRequest.groovy

@@ -2,4 +2,5 @@ package com.muwire.core.upload
 
 class ContentRequest extends Request {
     Range range
+    int have
 }

core/src/main/groovy/com/muwire/core/upload/ContentUploader.groovy

@@ -5,34 +5,58 @@ import java.nio.channels.FileChannel
 import java.nio.charset.StandardCharsets
 import java.nio.file.Files
 import java.nio.file.StandardOpenOption
+import java.util.stream.Collectors
 
+import com.muwire.core.Persona
 import com.muwire.core.connection.Endpoint
+import com.muwire.core.mesh.Mesh
+import com.muwire.core.util.DataUtil
+
+import net.i2p.data.Destination
 
 class ContentUploader extends Uploader {
     
     private final File file
     private final ContentRequest request
+    private final Mesh mesh
+    private final int pieceSize
     
-    ContentUploader(File file, ContentRequest request, Endpoint endpoint) {
+    ContentUploader(File file, ContentRequest request, Endpoint endpoint, Mesh mesh, int pieceSize) {
         super(endpoint)
         this.file = file
         this.request = request
+        this.mesh = mesh
+        this.pieceSize = pieceSize
     }
     
     @Override
     void respond() {
         OutputStream os = endpoint.getOutputStream()
         Range range = request.getRange()
-        if (range.start >= file.length() || range.end >= file.length()) {
-            os.write("416 Range Not Satisfiable\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
+        boolean satisfiable = true
+        final long length = file.length()
+        if (range.start >= length || range.end >= length)
+            satisfiable = false
+        if (satisfiable) {
+            int startPiece = range.start / (0x1 << pieceSize)
+            int endPiece = range.end / (0x1 << pieceSize)
+            for (int i = startPiece; i <= endPiece; i++)
+                satisfiable &= mesh.pieces.isDownloaded(i)
+        }
+        if (!satisfiable) {
+            os.write("416 Range Not Satisfiable\r\n".getBytes(StandardCharsets.US_ASCII))
+            writeMesh(request.downloader)
+            os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
             os.flush()
             return
         }
 
         os.write("200 OK\r\n".getBytes(StandardCharsets.US_ASCII))
-        os.write("Content-Range: $range.start-$range.end\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
-
-        FileChannel channel
+        os.write("Content-Range: $range.start-$range.end\r\n".getBytes(StandardCharsets.US_ASCII))
+        writeMesh(request.downloader)
+        os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
+        
+        FileChannel channel = null
         try {
             channel = Files.newByteChannel(file.toPath(), EnumSet.of(StandardOpenOption.READ))
             mapped = channel.map(FileChannel.MapMode.READ_ONLY, range.start, range.end - range.start + 1)
@@ -48,6 +72,21 @@ class ContentUploader extends Uploader {
         } finally {
             try {channel?.close() } catch (IOException ignored) {}
             endpoint.getOutputStream().flush()
+            synchronized(this) {
+                DataUtil.tryUnmap(mapped)
+                mapped = null
+            }
+        }
+    }
+    
+    private void writeMesh(Persona toExclude) {
+        String xHave = DataUtil.encodeXHave(mesh.pieces.getDownloaded(), mesh.pieces.nPieces)
+        endpoint.getOutputStream().write("X-Have: $xHave\r\n".getBytes(StandardCharsets.US_ASCII))
+        
+        Set<Persona> sources = mesh.getRandom(3, toExclude)
+        if (!sources.isEmpty()) {
+            String xAlts = sources.stream().map({ it.toBase64() }).collect(Collectors.joining(","))
+            endpoint.getOutputStream().write("X-Alt: $xAlts\r\n".getBytes(StandardCharsets.US_ASCII))
         }
     }
 
@@ -70,4 +109,14 @@ class ContentUploader extends Uploader {
         request.downloader.getHumanReadableName()
     }
 
+    @Override
+    public int getDonePieces() {
+        return request.have;
+    }
+
+    @Override
+    public int getTotalPieces() {
+        return mesh.pieces.nPieces;
+    }
+
 }

core/src/main/groovy/com/muwire/core/upload/HashListUploader.groovy

@@ -50,6 +50,16 @@ class HashListUploader extends Uploader {
     public String getDownloader() {
         request.downloader.getHumanReadableName()
     }
+
+    @Override
+    public int getDonePieces() {
+        return 0;
+    }
+
+    @Override
+    public int getTotalPieces() {
+        return 1;
+    }
     
     
 }

core/src/main/groovy/com/muwire/core/upload/Request.groovy

@@ -5,6 +5,7 @@ import java.nio.charset.StandardCharsets
 import com.muwire.core.Constants
 import com.muwire.core.InfoHash
 import com.muwire.core.Persona
+import com.muwire.core.util.DataUtil
 
 import groovy.util.logging.Log
 import net.i2p.data.Base64
@@ -48,8 +49,14 @@ class Request {
             def decoded = Base64.decode(encoded)
             downloader = new Persona(new ByteArrayInputStream(decoded))
         }
+        
+        int have = 0
+        if (headers.containsKey("X-Have")) {
+            def encoded = headers["X-Have"].trim()
+            have = DataUtil.decodeXHave(encoded).size()
+        }
         new ContentRequest( infoHash : infoHash, range : new Range(start, end), 
-            headers : headers, downloader : downloader)
+            headers : headers, downloader : downloader, have : have)
     }
     
     static Request parseHashListRequest(InfoHash infoHash, InputStream is) throws IOException {

core/src/main/groovy/com/muwire/core/upload/UploadManager.groovy

@@ -6,7 +6,12 @@ import com.muwire.core.EventBus
 import com.muwire.core.InfoHash
 import com.muwire.core.SharedFile
 import com.muwire.core.connection.Endpoint
+import com.muwire.core.download.DownloadManager
+import com.muwire.core.download.Downloader
+import com.muwire.core.download.SourceDiscoveredEvent
 import com.muwire.core.files.FileManager
+import com.muwire.core.mesh.Mesh
+import com.muwire.core.mesh.MeshManager
 
 import groovy.util.logging.Log
 import net.i2p.data.Base64
@@ -15,12 +20,17 @@ import net.i2p.data.Base64
 public class UploadManager {
     private final EventBus eventBus
     private final FileManager fileManager
+    private final MeshManager meshManager
+    private final DownloadManager downloadManager
     
     public UploadManager() {}
     
-    public UploadManager(EventBus eventBus, FileManager fileManager) {
+    public UploadManager(EventBus eventBus, FileManager fileManager, 
+        MeshManager meshManager, DownloadManager downloadManager) {
         this.eventBus = eventBus
         this.fileManager = fileManager
+        this.meshManager = meshManager
+        this.downloadManager = downloadManager
     }
     
     public void processGET(Endpoint e) throws IOException {
@@ -44,8 +54,10 @@ public class UploadManager {
             log.info("Responding to upload request for root $infoHashString")
 
             byte [] infoHashRoot = Base64.decode(infoHashString)
+            InfoHash infoHash = new InfoHash(infoHashRoot)
             Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot)
-            if (sharedFiles == null || sharedFiles.isEmpty()) {
+            Downloader downloader = downloadManager.downloaders.get(infoHash)
+            if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
                 log.info "file not found"
                 e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
                 e.getOutputStream().flush()
@@ -61,13 +73,31 @@ public class UploadManager {
                 return
             }
 
-            Request request = Request.parseContentRequest(new InfoHash(infoHashRoot), e.getInputStream())
+            ContentRequest request = Request.parseContentRequest(infoHash, e.getInputStream())
             if (request.downloader != null && request.downloader.destination != e.destination) {
                 log.info("Downloader persona doesn't match their destination")
                 e.close()
                 return
             }
-            Uploader uploader = new ContentUploader(sharedFiles.iterator().next().file, request, e)
+            
+            if (request.have > 0) 
+                eventBus.publish(new SourceDiscoveredEvent(infoHash : request.infoHash, source : request.downloader))
+            
+            Mesh mesh
+            File file
+            int pieceSize
+            if (downloader != null) {
+                mesh = meshManager.get(infoHash)
+                file = downloader.incompleteFile
+                pieceSize = downloader.pieceSizePow2
+            } else {
+                SharedFile sharedFile = sharedFiles.iterator().next();
+                mesh = meshManager.getOrCreate(request.infoHash, sharedFile.NPieces)
+                file = sharedFile.file
+                pieceSize = sharedFile.pieceSize
+            }
+                
+            Uploader uploader = new ContentUploader(file, request, e, mesh, pieceSize)
             eventBus.publish(new UploadEvent(uploader : uploader))
             try {
                 uploader.respond()
@@ -85,8 +115,10 @@ public class UploadManager {
         log.info("Responding to hashlist request for root $infoHashString")
         
         byte [] infoHashRoot = Base64.decode(infoHashString)
+        InfoHash infoHash = new InfoHash(infoHashRoot)
+        Downloader downloader = downloadManager.downloaders.get(infoHash)
         Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot)
-        if (sharedFiles == null || sharedFiles.isEmpty()) {
+        if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
             log.info "file not found"
             e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
             e.getOutputStream().flush()
@@ -102,13 +134,30 @@ public class UploadManager {
             return
         }
         
-        Request request = Request.parseHashListRequest(new InfoHash(infoHashRoot), e.getInputStream())
+        Request request = Request.parseHashListRequest(infoHash, e.getInputStream())
         if (request.downloader != null && request.downloader.destination != e.destination) {
             log.info("Downloader persona doesn't match their destination")
             e.close()
             return
         }
-        Uploader uploader = new HashListUploader(e, sharedFiles.iterator().next().infoHash, request)
+        
+        InfoHash fullInfoHash
+        if (downloader == null) {
+            fullInfoHash = sharedFiles.iterator().next().infoHash
+        } else {
+            byte [] hashList = downloader.getInfoHash().getHashList()
+            if (hashList != null && hashList.length > 0)
+                fullInfoHash = downloader.getInfoHash()
+            else {
+                log.info("infohash not found in downloader")
+                e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
+                e.getOutputStream().flush()
+                e.close()
+                return
+            }
+        }
+        
+        Uploader uploader = new HashListUploader(e, fullInfoHash, request)
         eventBus.publish(new UploadEvent(uploader : uploader))
         try {
             uploader.respond()
@@ -130,8 +179,10 @@ public class UploadManager {
             log.info("Responding to upload request for root $infoHashString")
 
             infoHashRoot = Base64.decode(infoHashString)
+            infoHash = new InfoHash(infoHashRoot)
             sharedFiles = fileManager.getSharedFiles(infoHashRoot)
-            if (sharedFiles == null || sharedFiles.isEmpty()) {
+            downloader = downloadManager.downloaders.get(infoHash)
+            if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
                 log.info "file not found"
                 e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
                 e.getOutputStream().flush()
@@ -153,7 +204,25 @@ public class UploadManager {
                 e.close()
                 return
             }
-            uploader = new ContentUploader(sharedFiles.iterator().next().file, request, e)
+            
+            if (request.have > 0)
+                eventBus.publish(new SourceDiscoveredEvent(infoHash : request.infoHash, source : request.downloader))
+            
+            Mesh mesh
+            File file
+            int pieceSize
+            if (downloader != null) {
+                mesh = meshManager.get(infoHash)
+                file = downloader.incompleteFile
+                pieceSize = downloader.pieceSizePow2
+            } else {
+                SharedFile sharedFile = sharedFiles.iterator().next();
+                mesh = meshManager.getOrCreate(request.infoHash, sharedFile.NPieces)
+                file = sharedFile.file
+                pieceSize = sharedFile.pieceSize
+            }
+
+            uploader = new ContentUploader(file, request, e, mesh, pieceSize)
             eventBus.publish(new UploadEvent(uploader : uploader))
             try {
                 uploader.respond()

core/src/main/groovy/com/muwire/core/upload/Uploader.groovy

@@ -32,4 +32,8 @@ abstract class Uploader {
     abstract int getProgress();
     
     abstract String getDownloader();
+    
+    abstract int getDonePieces();
+    
+    abstract int getTotalPieces()
 }

core/src/main/groovy/com/muwire/core/util/DataUtil.groovy

@@ -1,9 +1,14 @@
 package com.muwire.core.util
 
+import java.lang.reflect.Field
+import java.lang.reflect.Method
+import java.nio.ByteBuffer
 import java.nio.charset.StandardCharsets
 
 import com.muwire.core.Constants
 
+import net.i2p.data.Base64
+
 class DataUtil {
 	
 	private final static int MAX_SHORT = (0x1 << 16) - 1
@@ -79,4 +84,73 @@ class DataUtil {
         }
         new String(baos.toByteArray(), StandardCharsets.US_ASCII)
     }
+    
+    public static String encodeXHave(List<Integer> pieces, int totalPieces) {
+        int bytes = totalPieces / 8
+        if (totalPieces % 8 != 0)
+            bytes++
+        byte[] raw = new byte[bytes]
+        pieces.each {
+            int byteIdx = it / 8
+            int offset = it % 8
+            int mask = 0x80 >>> offset
+            raw[byteIdx] |= mask
+        }
+        Base64.encode(raw)
+    }
+    
+    public static List<Integer> decodeXHave(String xHave) {
+        byte [] availablePieces = Base64.decode(xHave)
+        List<Integer> available = new ArrayList<>()
+        availablePieces.eachWithIndex {b, i ->
+            for (int j = 0; j < 8 ; j++) {
+                byte mask = 0x80 >>> j
+                if ((b & mask) == mask) {
+                    available.add(i * 8 + j)
+                }
+            }
+        }
+        available
+    }
+    
+    public static Exception findRoot(Exception e) {
+        while(e.getCause() != null)
+            e = e.getCause()
+        e
+    }
+    
+    public static void tryUnmap(ByteBuffer cb) {
+        if (cb==null || !cb.isDirect()) return;
+        // we could use this type cast and call functions without reflection code,
+        // but static import from sun.* package is risky for non-SUN virtual machine.
+        //try { ((sun.nio.ch.DirectBuffer)cb).cleaner().clean(); } catch (Exception ex) { }
+    
+        // JavaSpecVer: 1.6, 1.7, 1.8, 9, 10
+        boolean isOldJDK = System.getProperty("java.specification.version","99").startsWith("1.");
+        try {
+            if (isOldJDK) {
+                Method cleaner = cb.getClass().getMethod("cleaner");
+                cleaner.setAccessible(true);
+                Method clean = Class.forName("sun.misc.Cleaner").getMethod("clean");
+                clean.setAccessible(true);
+                clean.invoke(cleaner.invoke(cb));
+            } else {
+                Class unsafeClass;
+                try {
+                    unsafeClass = Class.forName("sun.misc.Unsafe");
+                } catch(Exception ex) {
+                    // jdk.internal.misc.Unsafe doesn't yet have an invokeCleaner() method,
+                    // but that method should be added if sun.misc.Unsafe is removed.
+                    unsafeClass = Class.forName("jdk.internal.misc.Unsafe");
+                }
+                Method clean = unsafeClass.getMethod("invokeCleaner", ByteBuffer.class);
+                clean.setAccessible(true);
+                Field theUnsafeField = unsafeClass.getDeclaredField("theUnsafe");
+                theUnsafeField.setAccessible(true);
+                Object theUnsafe = theUnsafeField.get(null);
+                clean.invoke(theUnsafe, cb);
+            }
+        } catch(Exception ex) { }
+        cb = null;
+    }
 }

core/src/main/java/com/muwire/core/DownloadedFile.java

@@ -1,6 +1,7 @@
 package com.muwire.core;
 
 import java.io.File;
+import java.io.IOException;
 import java.util.Set;
 
 import net.i2p.data.Destination;
@@ -9,7 +10,8 @@ public class DownloadedFile extends SharedFile {
 	
 	private final Set<Destination> sources;
 
-	public DownloadedFile(File file, InfoHash infoHash, int pieceSize, Set<Destination> sources) {
+	public DownloadedFile(File file, InfoHash infoHash, int pieceSize, Set<Destination> sources) 
+	throws IOException {
 		super(file, infoHash, pieceSize);
 		this.sources = sources;
 	}

core/src/main/java/com/muwire/core/SharedFile.java

@@ -1,6 +1,7 @@
 package com.muwire.core;
 
 import java.io.File;
+import java.io.IOException;
 
 public class SharedFile {
 
@@ -8,10 +9,15 @@ public class SharedFile {
 	private final InfoHash infoHash;
 	private final int pieceSize;
 	
-	public SharedFile(File file, InfoHash infoHash, int pieceSize) {
+	private final String cachedPath;
+	private final long cachedLength;
+	
+	public SharedFile(File file, InfoHash infoHash, int pieceSize) throws IOException {
 		this.file = file;
 		this.infoHash = infoHash;
 		this.pieceSize = pieceSize;
+		this.cachedPath = file.getAbsolutePath();
+		this.cachedLength = file.length();
 	}
 
 	public File getFile() {
@@ -26,6 +32,23 @@ public class SharedFile {
 	    return pieceSize;
 	}
 	
+	public int getNPieces() {
+	    long length = file.length();
+	    int rawPieceSize = 0x1 << pieceSize;
+	    int rv = (int) (length / rawPieceSize);
+	    if (length % rawPieceSize != 0)
+	        rv++;
+	    return rv;
+	}
+	
+	public String getCachedPath() {
+	    return cachedPath;
+	}
+	
+	public long getCachedLength() {
+	    return cachedLength;
+	}
+	
 	@Override
 	public int hashCode() {
 	    return file.hashCode() ^ infoHash.hashCode();

core/src/test/groovy/com/muwire/core/download/DownloadSessionTest.groovy

@@ -1,21 +1,30 @@
 package com.muwire.core.download
 
+import static org.junit.Assert.fail
+
 import org.junit.After
+import org.junit.Before
 import org.junit.Test
 
+import com.muwire.core.EventBus
 import com.muwire.core.InfoHash
+import com.muwire.core.Persona
+import com.muwire.core.Personas
 import com.muwire.core.connection.Endpoint
 import com.muwire.core.files.FileHasher
 import static com.muwire.core.util.DataUtil.readTillRN
+import static com.muwire.core.util.DataUtil.encodeXHave
 
 import net.i2p.data.Base64
+import net.i2p.util.ConcurrentHashSet
 
 class DownloadSessionTest {
     
+    private EventBus eventBus
     private File source, target
     private InfoHash infoHash
     private Endpoint endpoint
-    private Pieces pieces, claimed
+    private Pieces pieces
     private String rootBase64
     
     private DownloadSession session
@@ -24,6 +33,16 @@ class DownloadSessionTest {
     private InputStream fromDownloader, fromUploader
     private OutputStream toDownloader, toUploader
     
+    private volatile boolean performed
+    private Set<Integer> available = new ConcurrentHashSet<>()
+    private volatile IOException thrown
+    
+    
+    @Before
+    public void setUp() {
+        eventBus = new EventBus()
+    }
+    
     private void initSession(int size, def claimedPieces = []) {
         Random r = new Random()
         byte [] content = new byte[size]
@@ -48,8 +67,7 @@ class DownloadSessionTest {
         else
             nPieces = size / pieceSize + 1
         pieces = new Pieces(nPieces)
-        claimed = new Pieces(nPieces)
-        claimedPieces.each {claimed.markDownloaded(it)}
+        claimedPieces.each {pieces.claimed.set(it)}
         
         fromDownloader = new PipedInputStream()
         fromUploader = new PipedInputStream()
@@ -57,12 +75,20 @@ class DownloadSessionTest {
         toUploader = new PipedOutputStream(fromDownloader)
         endpoint = new Endpoint(null, fromUploader, toUploader, null)
         
-        session = new DownloadSession("",pieces, claimed, infoHash, endpoint, target, pieceSize, size)
-        downloadThread = new Thread( { session.request() } as Runnable)
+        session = new DownloadSession(eventBus, "",pieces, infoHash, endpoint, target, pieceSize, size, available)
+        downloadThread = new Thread( { perform() } as Runnable)
         downloadThread.setDaemon(true)
         downloadThread.start()
     }
     
+    private void perform() {
+        try { 
+            performed = session.request()
+        } catch (IOException e) {
+            thrown = e
+        }
+    }
+    
     @After
     public void teardown() {
         source?.delete()
@@ -77,6 +103,7 @@ class DownloadSessionTest {
         assert "GET $rootBase64" == readTillRN(fromDownloader)
         assert "Range: 0-19" == readTillRN(fromDownloader)
         readTillRN(fromDownloader)
+        readTillRN(fromDownloader)
         assert "" == readTillRN(fromDownloader)
         
         toDownloader.write("200 OK\r\n".bytes)
@@ -88,6 +115,9 @@ class DownloadSessionTest {
         
         assert pieces.isComplete()
         assert target.bytes == source.bytes
+        assert performed
+        assert available.isEmpty()
+        assert thrown == null
     }
     
     @Test
@@ -99,6 +129,7 @@ class DownloadSessionTest {
         assert "GET $rootBase64" == readTillRN(fromDownloader)
         readTillRN(fromDownloader)
         readTillRN(fromDownloader)
+        readTillRN(fromDownloader)
         assert "" == readTillRN(fromDownloader)
         
         toDownloader.write("200 OK\r\n".bytes)
@@ -109,6 +140,9 @@ class DownloadSessionTest {
         Thread.sleep(150)
         assert pieces.isComplete()
         assert target.bytes == source.bytes
+        assert performed
+        assert available.isEmpty()
+        assert thrown == null
     }
     
     @Test
@@ -126,6 +160,7 @@ class DownloadSessionTest {
         assert (start == 0 && end == ((1 << pieceSize) - 1)) || 
             (start == (1 << pieceSize) && end == (1 << pieceSize))
         
+        readTillRN(fromDownloader)
         readTillRN(fromDownloader)
         assert "" == readTillRN(fromDownloader)
         
@@ -139,6 +174,9 @@ class DownloadSessionTest {
         Thread.sleep(150)
         assert !pieces.isComplete()
         assert 1 == pieces.donePieces()
+        assert performed
+        assert available.isEmpty()
+        assert thrown == null
     }
     
     @Test
@@ -146,7 +184,10 @@ class DownloadSessionTest {
         initSession(20, [0])
         long now = System.currentTimeMillis()
         downloadThread.join(100)
-        assert 100 > (System.currentTimeMillis() - now)
+        assert 100 >= (System.currentTimeMillis() - now)
+        assert !performed
+        assert available.isEmpty()
+        assert thrown == null
     }
     
     @Test
@@ -154,7 +195,7 @@ class DownloadSessionTest {
         int pieceSize = FileHasher.getPieceSize(1)
         int size = (1 << pieceSize) * 10
         initSession(size, [1,2,3,4,5,6,7,8,9])
-        assert !claimed.isMarked(0)
+        assert !pieces.claimed.get(0)
         
         assert "GET $rootBase64" == readTillRN(fromDownloader)
         String range = readTillRN(fromDownloader)
@@ -162,7 +203,131 @@ class DownloadSessionTest {
         int start = Integer.parseInt(matcher[0][1])
         int end = Integer.parseInt(matcher[0][2])
         
-        assert claimed.isMarked(0)
+        assert pieces.claimed.get(0)
         assert start == 0 && end == (1 << pieceSize) - 1
     }
+    
+    @Test
+    public void test416NoHave() {
+        initSession(20)
+        readAllHeaders(fromDownloader)
+        
+        toDownloader.write("416 don't have it\r\n\r\n".bytes)
+        toDownloader.flush()
+        Thread.sleep(150)
+        assert !performed
+        assert available.isEmpty()
+        assert thrown != null
+    }
+    
+    @Test
+    public void test416Have() {
+        initSession(20)
+        readAllHeaders(fromDownloader)
+        
+        toDownloader.write("416 don't have it\r\n".bytes)
+        toDownloader.write("X-Have: ${encodeXHave([0], 1)}\r\n\r\n".bytes)
+        toDownloader.flush()
+        
+        Thread.sleep(150)
+        assert performed
+        assert available.contains(0)
+        assert thrown == null
+    }
+    
+    @Test
+    public void test416Have2Pieces() {
+        int pieceSize = FileHasher.getPieceSize(1)
+        int size = (1 << pieceSize) + 1
+        initSession(size)
+        readAllHeaders(fromDownloader)
+        
+        toDownloader.write("416 don't have it\r\n".bytes)
+        toDownloader.write("X-Have: ${encodeXHave([1], 2)}\r\n\r\n".bytes)
+        toDownloader.flush()
+        
+        Thread.sleep(150)
+        assert performed
+        assert available.contains(1)
+        assert thrown == null
+    }
+    
+    @Test
+    public void test200TwoPieces1Available() {
+        int pieceSize = FileHasher.getPieceSize(1)
+        int size = (1 << pieceSize) * 9 + 1
+        initSession(size)
+        
+        Set<String> headers = readAllHeaders(fromDownloader)
+        def matcher = null
+        headers.each {
+            if (it.startsWith("Range"))
+                matcher = (it =~ /^Range: (\d+)-(\d+)$/)
+        }
+        assert matcher.groupCount() > 0
+        int start = Integer.parseInt(matcher[0][1])
+        int end = Integer.parseInt(matcher[0][2])
+        
+        if (start == 0) 
+            fail("inconlcusive")
+        
+        toDownloader.write("416 don't have it \r\n".bytes)
+        toDownloader.write("X-Have: ${encodeXHave([0],2)}\r\n\r\n".bytes)
+        toDownloader.flush()
+        downloadThread.join()
+        
+        assert performed
+        performed = false
+        assert available.contains(0)
+        assert thrown == null
+        
+        // request same session
+        downloadThread = new Thread( { perform() } as Runnable)
+        downloadThread.setDaemon(true)
+        downloadThread.start()
+        
+        Thread.sleep(150)
+        
+        headers = readAllHeaders(fromDownloader)
+        matcher = null
+        headers.each {
+            if (it.startsWith("Range"))
+            matcher = (it =~ /^Range: (\d+)-(\d+)$/)
+        }
+        assert matcher.groupCount() > 0
+        start = Integer.parseInt(matcher[0][1])
+        end = Integer.parseInt(matcher[0][2])
+        assert start == 0
+    }
+    
+    @Test
+    public void testXAlt() throws Exception {
+        Personas personas = new Personas()
+        def sources = []
+        def listener = new Object() {
+            public void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
+                sources << e.source
+            }
+        }
+        eventBus.register(SourceDiscoveredEvent.class, listener)
+        
+        initSession(20)
+        readAllHeaders(fromDownloader)
+        toDownloader.write("416 don't have it\r\n".bytes)
+        toDownloader.write("X-Alt: ${personas.persona1.toBase64()},${personas.persona2.toBase64()}\r\n\r\n".bytes)
+        toDownloader.flush()
+        
+        Thread.sleep(150)
+        assert sources.contains(personas.persona1)
+        assert sources.contains(personas.persona2)
+        assert 2 == sources.size()
+    }
+    
+    private static Set<String> readAllHeaders(InputStream is) {
+        Set<String> rv = new HashSet<>()
+        String header
+        while((header = readTillRN(is)) != "")
+            rv.add(header)
+        rv
+    }
 }

core/src/test/groovy/com/muwire/core/download/PiecesTest.groovy

@@ -16,7 +16,7 @@ class PiecesTest {
     public void testSinglePiece() {
         pieces = new Pieces(1)
         assert !pieces.isComplete()
-        assert pieces.getRandomPiece() == 0
+        assert pieces.claim() == 0
         pieces.markDownloaded(0)
         assert pieces.isComplete()
     }
@@ -25,13 +25,28 @@ class PiecesTest {
     public void testTwoPieces() {
         pieces = new Pieces(2)
         assert !pieces.isComplete()
-        int piece = pieces.getRandomPiece()
+        int piece = pieces.claim()
         assert piece == 0 || piece == 1
         pieces.markDownloaded(piece)
         assert !pieces.isComplete()
-        int piece2 = pieces.getRandomPiece()
+        int piece2 = pieces.claim()
         assert piece != piece2
         pieces.markDownloaded(piece2)
         assert pieces.isComplete()
     }
+    
+    @Test
+    public void testClaimAvailable() {
+        pieces = new Pieces(2)
+        int claimed = pieces.claim([0].toSet())
+        assert claimed == 0
+        assert -1 == pieces.claim([0].toSet())
+    }
+    
+    @Test
+    public void testClaimNoneAvailable() {
+        pieces = new Pieces(20)
+        int claimed = pieces.claim()
+        assert -1 == pieces.claim([claimed].toSet())
+    }
 }

core/src/test/groovy/com/muwire/core/files/FileHasherTest.groovy

@@ -26,7 +26,7 @@ class FileHasherTest extends GroovyTestCase {
 	void testPieceSize() {
 		assert 17 == FileHasher.getPieceSize(1000000)
 		assert 17 == FileHasher.getPieceSize(100000000)
-		assert 27 == FileHasher.getPieceSize(FileHasher.MAX_SIZE)
+		assert 24 == FileHasher.getPieceSize(FileHasher.MAX_SIZE)
 		shouldFail IllegalArgumentException, {
 			FileHasher.getPieceSize(Long.MAX_VALUE)
 		}

core/src/test/groovy/com/muwire/core/files/HasherServiceTest.groovy

@@ -27,6 +27,7 @@ class HasherServiceTest {
 		hasher = new FileHasher()
 		service = new HasherService(hasher, eventBus, new FileManager(eventBus, new MuWireSettings()))
         eventBus.register(FileHashedEvent.class, listener)
+        eventBus.register(FileSharedEvent.class, service)
 		service.start()
 	}
 	

core/src/test/groovy/com/muwire/core/files/PersisterServiceLoadingTest.groovy

@@ -78,7 +78,7 @@ class PersisterServiceLoadingTest {
 		persisted.write json
 		
 		PersisterService ps = new PersisterService(persisted, eventBus, 100, null)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(2000)
 		
 		assert listener.publishedFiles.size() == 1
@@ -121,7 +121,7 @@ class PersisterServiceLoadingTest {
 		persisted.write json
 		
 		PersisterService ps = new PersisterService(persisted, eventBus, 100, null)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(2000)
 		
 		assert listener.publishedFiles.size() == 1
@@ -163,7 +163,7 @@ class PersisterServiceLoadingTest {
 		persisted.append "$json2\n"
 		
 		PersisterService ps = new PersisterService(persisted, eventBus, 100, null)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(2000)
 		
 		assert listener.publishedFiles.size() == 2
@@ -195,7 +195,7 @@ class PersisterServiceLoadingTest {
 		persisted.write json1
 		
 		PersisterService ps = new PersisterService(persisted, eventBus, 100, null)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(2000)
 		
 		assert listener.publishedFiles.size() == 1

core/src/test/groovy/com/muwire/core/files/PersisterServiceSavingTest.groovy

@@ -58,7 +58,7 @@ class PersisterServiceSavingTest {
 		sf = new SharedFile(f, ih, 0)
 		
 		ps = new PersisterService(persisted, eventBus, 100, fileSource)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(1500)
 	
 		JsonSlurper jsonSlurper = new JsonSlurper()	
@@ -77,7 +77,7 @@ class PersisterServiceSavingTest {
 		sf = new DownloadedFile(f, ih, 0, new HashSet([dests.dest1, dests.dest2]))
 		
 		ps = new PersisterService(persisted, eventBus, 100, fileSource)
-		ps.start()
+		ps.onUILoadedEvent(null)
 		Thread.sleep(1500)
 	
 		JsonSlurper jsonSlurper = new JsonSlurper()

core/src/test/groovy/com/muwire/core/search/SearchIndexTest.groovy

@@ -83,4 +83,11 @@ class SearchIndexTest {
 		assert found.size() == 1
 		assert found.contains("b c.d")
 	}
+    
+    @Test
+    void testDuplicateTerm() {
+        initIndex(["MuWire-0.3.3.jar"])
+        def found = index.search(["muwire", "0", "3", "jar"])
+        assert found.size() == 1
+    }
 }

core/src/test/groovy/com/muwire/core/upload/RequestParsingTest.groovy

@@ -9,18 +9,18 @@ import com.muwire.core.InfoHash
 
 class RequestParsingTest {
     
-    Request request
+    ContentRequest request
     
     private void fromString(String requestString) {
         def is = new ByteArrayInputStream(requestString.getBytes(StandardCharsets.US_ASCII))
-        request = Request.parse(new InfoHash(new byte[InfoHash.SIZE]), is)
+        request = Request.parseContentRequest(new InfoHash(new byte[InfoHash.SIZE]), is)
     }
 
     
     private static void failed(String requestString) {
         try {
             def is = new ByteArrayInputStream(requestString.getBytes(StandardCharsets.US_ASCII))
-            Request.parse(new InfoHash(new byte[InfoHash.SIZE]), is)
+            Request.parseContentRequest(new InfoHash(new byte[InfoHash.SIZE]), is)
             assert false
         } catch (IOException expected) {}
     }

core/src/test/groovy/com/muwire/core/upload/UploaderTest.groovy

@@ -19,7 +19,7 @@ class UploaderTest {
     InputStream is
     OutputStream os
     
-    Request request
+    ContentRequest request
     Uploader uploader
     
     byte[] inFile
@@ -52,7 +52,7 @@ class UploaderTest {
     }
     
     private void startUpload() {
-        uploader = new Uploader(file, request, endpoint)
+        uploader = new ContentUploader(file, request, endpoint)
         uploadThread = new Thread(uploader.respond() as Runnable)
         uploadThread.setDaemon(true)
         uploadThread.start()
@@ -77,7 +77,7 @@ class UploaderTest {
     @Test
     public void testSmallFile() {
         fillFile(20)
-        request = new Request(range : new Range(0,19))
+        request = new ContentRequest(range : new Range(0,19))
         startUpload()
         assert "200 OK" == readUntilRN()
         assert "Content-Range: 0-19" == readUntilRN()
@@ -92,7 +92,7 @@ class UploaderTest {
     @Test
     public void testRequestMiddle() {
         fillFile(20)
-        request = new Request(range : new Range(5,15))
+        request = new ContentRequest(range : new Range(5,15))
         startUpload()
         assert "200 OK" == readUntilRN()
         assert "Content-Range: 5-15" == readUntilRN()
@@ -108,7 +108,7 @@ class UploaderTest {
     @Test
     public void testOutOfRange() {
         fillFile(20)
-        request = new Request(range : new Range(0,20))
+        request = new ContentRequest(range : new Range(0,20))
         startUpload()
         assert "416 Range Not Satisfiable" == readUntilRN()
         assert "" == readUntilRN()
@@ -118,7 +118,7 @@ class UploaderTest {
     public void testLargeFile() {
         final int length = 0x1 << 14
         fillFile(length)
-        request = new Request(range : new Range(0, length - 1))
+        request = new ContentRequest(range : new Range(0, length - 1))
         startUpload()
         readUntilRN()
         readUntilRN()

doc/design.md

@@ -49,7 +49,7 @@ Files are transferred over HTTP1.1 protocol with some custom headers added for d
 
 ### Mesh management
 
-Download mesh management is identical to Gnutella, except instead of ip addresses MuWire personas are used.  [More information](http://rfc-gnutella.sourceforge.net/developer/tmp/download-mesh.html) 
+Download mesh management is a simplified version of Gnutella's "Alternate Location" system.  For more information see the "download-mesh" document.
 
 ### In-Network updates
 

doc/download-mesh.md

@@ -0,0 +1,15 @@
+# Download Mesh / Partial Sharing
+
+MuWire uses a system similar to Gnutella's "Alternate Location" download mesh management system, however it is simplified to account for I2P's strengths and borrows a bit from BitTorrent's "Have" message.
+
+### "X-Have" header
+
+With every request a downloader makes it sends an "X-Have" header containing the Base64-encoded representation of a bitfield where bits set to 1 represent pieces of the file that the downloader already has.  To make partial file sharing possible, if the uploader does not have the complete file it also sends this header in every response.  If the header is missing it is assumed the uploader has the complete file.
+
+### "X-Alt" header
+
+The uploader can recommend other uploaders to the downloader via the "X-Alt" header.  The format of this header is a comma-separated list of Base64-encoded Personas that have previously reported having at least one piece of the file to the uploader via the "X-Have" header.
+
+### Differences from Gnutella
+
+Unlike Gnutella the uploader is the sole repository where possible sources of the file are tracked.  There is no negative "X-Nalt" header to prevent attacking the download mesh by mass downvoting of sources.

gradle.properties

@@ -1,5 +1,5 @@
 group         = com.muwire
-version       = 0.2.9
+version       = 0.4.5
 groovyVersion = 2.4.15
 slf4jVersion  = 1.7.25
 spockVersion  = 1.1-groovy-2.4

gui/griffon-app/conf/Config.groovy

@@ -26,4 +26,14 @@ mvcGroups {
         view = 'com.muwire.gui.OptionsView'
         controller = 'com.muwire.gui.OptionsController'
     }
+    "mu-wire-status" {
+        model = 'com.muwire.gui.MuWireStatusModel'
+        view = 'com.muwire.gui.MuWireStatusView'
+        controller = 'com.muwire.gui.MuWireStatusController'
+    }
+    'i-2-p-status' {
+        model = 'com.muwire.gui.I2PStatusModel'
+        view = 'com.muwire.gui.I2PStatusView'
+        controller = 'com.muwire.gui.I2PStatusController'
+    }
 }

gui/griffon-app/controllers/com/muwire/gui/I2PStatusController.groovy

@@ -0,0 +1,41 @@
+package com.muwire.gui
+
+import griffon.core.artifact.GriffonController
+import griffon.core.controller.ControllerAction
+import griffon.inject.MVCMember
+import griffon.metadata.ArtifactProviderFor
+import net.i2p.router.Router
+
+import javax.annotation.Nonnull
+
+import com.muwire.core.Core
+
+@ArtifactProviderFor(GriffonController)
+class I2PStatusController {
+    @MVCMember @Nonnull
+    I2PStatusModel model
+    @MVCMember @Nonnull
+    I2PStatusView view
+
+    @ControllerAction
+    void refresh() {
+        Core core = application.context.get("core")
+        Router router = core.router
+        model.networkStatus = router._context.commSystem().status.toStatusString()
+        model.floodfill = router._context.netDb().floodfillEnabled()
+        model.ntcpConnections = router._context.commSystem().getTransports()["NTCP"].countPeers()
+        model.ssuConnections = router._context.commSystem().getTransports()["SSU"].countPeers()
+        model.participatingTunnels = router._context.tunnelManager().getParticipatingCount()
+        model.activePeers = router._context.profileOrganizer().countActivePeers()
+        model.receiveBps = router._context.bandwidthLimiter().getReceiveBps15s()
+        model.sendBps = router._context.bandwidthLimiter().getSendBps15s()
+        model.participatingBW = router._context.bandwidthLimiter().getCurrentParticipatingBandwidth()
+        
+    }
+    
+    @ControllerAction
+    void close() {
+        view.dialog.setVisible(false)
+        mvcGroup.destroy()
+    }
+}

gui/griffon-app/controllers/com/muwire/gui/MainFrameController.groovy

@@ -14,9 +14,14 @@ import javax.inject.Inject
 
 import com.muwire.core.Constants
 import com.muwire.core.Core
+import com.muwire.core.SharedFile
 import com.muwire.core.download.DownloadStartedEvent
 import com.muwire.core.download.UIDownloadCancelledEvent
 import com.muwire.core.download.UIDownloadEvent
+import com.muwire.core.download.UIDownloadPausedEvent
+import com.muwire.core.download.UIDownloadResumedEvent
+import com.muwire.core.files.DirectoryUnsharedEvent
+import com.muwire.core.files.FileUnsharedEvent
 import com.muwire.core.search.QueryEvent
 import com.muwire.core.search.SearchEvent
 import com.muwire.core.trust.TrustEvent
@@ -30,6 +35,8 @@ class MainFrameController {
     
     @MVCMember @Nonnull
     MainFrameModel model
+    @MVCMember @Nonnull
+    MainFrameView view
 
     private volatile Core core
     
@@ -42,6 +49,8 @@ class MainFrameController {
         search = search.trim()
         if (search.length() == 0)
             return
+        if (search.length() > 128)
+            search = search.substring(0,128)
         def uuid = UUID.randomUUID()
         Map<String, Object> params = new HashMap<>()
         params["search-terms"] = search
@@ -67,7 +76,9 @@ class MainFrameController {
             // this can be improved a lot
             def replaced = search.toLowerCase().trim().replaceAll(Constants.SPLIT_PATTERN, " ")
             def terms = replaced.split(" ")
-            searchEvent = new SearchEvent(searchTerms : terms, uuid : uuid, oobInfohash: true)
+            def nonEmpty = []
+            terms.each { if (it.length() > 0) nonEmpty << it }
+            searchEvent = new SearchEvent(searchTerms : nonEmpty, uuid : uuid, oobInfohash: true)
         }
         core.eventBus.publish(new QueryEvent(searchEvent : searchEvent, firstHop : true, 
             replyTo: core.me.destination, receivedOn: core.me.destination,
@@ -162,6 +173,14 @@ class MainFrameController {
     void resume() {
         def downloader = model.downloads[selectedDownload()].downloader
         downloader.resume()
+        core.eventBus.publish(new UIDownloadResumedEvent())
+    }
+    
+    @ControllerAction
+    void pause() {
+        def downloader = model.downloads[selectedDownload()].downloader
+        downloader.pause()
+        core.eventBus.publish(new UIDownloadPausedEvent())
     }
 
     private void markTrust(String tableName, TrustLevel level, def list) {
@@ -191,8 +210,23 @@ class MainFrameController {
         markTrust("trusted-table", TrustLevel.NEUTRAL, model.trusted)
     }
     
-    void unshareSelectedFiles() {
-        println "unsharing selected files"
+    void unshareSelectedFile() {
+        SharedFile sf = view.selectedSharedFile()
+        if (sf == null)
+            return
+        core.eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
+    }
+    
+    void stopWatchingDirectory() {
+        String directory = mvcGroup.view.getSelectedWatchedDirectory()
+        if (directory == null)
+            return
+        core.muOptions.watchedDirectories.remove(directory)
+        saveMuWireSettings()    
+        core.eventBus.publish(new DirectoryUnsharedEvent(directory : new File(directory)))
+                
+        model.watched.remove(directory)
+        builder.getVariable("watched-directories-table").model.fireTableDataChanged()
     }
     
     void saveMuWireSettings() {

gui/griffon-app/controllers/com/muwire/gui/MuWireStatusController.groovy

@@ -0,0 +1,45 @@
+package com.muwire.gui
+
+import griffon.core.artifact.GriffonController
+import griffon.core.controller.ControllerAction
+import griffon.inject.MVCMember
+import griffon.metadata.ArtifactProviderFor
+import javax.annotation.Nonnull
+
+import com.muwire.core.Core
+
+@ArtifactProviderFor(GriffonController)
+class MuWireStatusController {
+    @MVCMember @Nonnull
+    MuWireStatusModel model
+    @MVCMember @Nonnull
+    MuWireStatusView view
+
+    @ControllerAction
+    void refresh() {
+        Core core = application.context.get("core")
+        
+        int incoming = 0
+        int outgoing = 0
+        core.connectionManager.getConnections().each {
+            if (it.incoming)
+                incoming++
+            else
+                outgoing++
+        }
+        model.incomingConnections = incoming
+        model.outgoingConnections = outgoing
+        
+        model.knownHosts = core.hostCache.hosts.size()
+        
+        model.sharedFiles = core.fileManager.fileToSharedFile.size()
+        
+        model.downloads = core.downloadManager.downloaders.size()
+    }
+    
+    @ControllerAction
+    void close() {
+        view.dialog.setVisible(false)
+        mvcGroup.destroy()
+    }
+}

gui/griffon-app/controllers/com/muwire/gui/OptionsController.groovy

@@ -4,9 +4,15 @@ import griffon.core.artifact.GriffonController
 import griffon.core.controller.ControllerAction
 import griffon.inject.MVCMember
 import griffon.metadata.ArtifactProviderFor
+import groovy.util.logging.Log
+
+import java.util.logging.Level
+
 import javax.annotation.Nonnull
+import javax.swing.JFileChooser
 
 import com.muwire.core.Core
+import com.muwire.core.MuWireSettings
 
 @ArtifactProviderFor(GriffonController)
 class OptionsController {
@@ -19,6 +25,7 @@ class OptionsController {
     void save() {
         String text
         Core core = application.context.get("core")
+        MuWireSettings settings = application.context.get("muwire-settings")
         
         def i2pProps = core.i2pOptions
         
@@ -38,6 +45,17 @@ class OptionsController {
         model.outboundLength = text
         i2pProps["outbound.length"] = text
         
+        if (settings.embeddedRouter) {
+            text = view.i2pNTCPPortField.text
+            model.i2pNTCPPort = text
+            i2pProps["i2np.ntcp.port"] = text
+            
+            text = view.i2pUDPPortField.text
+            model.i2pUDPPort = text
+            i2pProps["i2np.udp.port"] = text
+        }
+        
+        
         File i2pSettingsFile = new File(core.home, "i2p.properties")
         i2pSettingsFile.withOutputStream { 
             i2pProps.store(it,"")
@@ -46,13 +64,16 @@ class OptionsController {
         text = view.retryField.text
         model.downloadRetryInterval = text
 
-        def settings = application.context.get("muwire-settings")
         settings.downloadRetryInterval = Integer.valueOf(text)
 
         text = view.updateField.text
         model.updateCheckInterval = text
         settings.updateCheckInterval = Integer.valueOf(text)
 
+        boolean autoDownloadUpdate = view.autoDownloadUpdateCheckbox.model.isSelected()
+        model.autoDownloadUpdate = autoDownloadUpdate
+        settings.autoDownloadUpdate = autoDownloadUpdate
+        
         boolean onlyTrusted = view.allowUntrustedCheckbox.model.isSelected()
         model.onlyTrusted = onlyTrusted
         settings.setAllowUntrusted(!onlyTrusted)
@@ -60,7 +81,19 @@ class OptionsController {
         boolean shareDownloaded = view.shareDownloadedCheckbox.model.isSelected()
         model.shareDownloadedFiles = shareDownloaded
         settings.shareDownloadedFiles = shareDownloaded
-                        
+        
+        String downloadLocation = model.downloadLocation
+        settings.downloadLocation = new File(downloadLocation)
+
+        if (settings.embeddedRouter) {
+            text = view.inBwField.text
+            model.inBw = text
+            settings.inBw = Integer.valueOf(text)
+            text = view.outBwField.text
+            model.outBw = text
+            settings.outBw = Integer.valueOf(text)
+        }
+                                
         File settingsFile = new File(core.home, "MuWire.properties")
         settingsFile.withOutputStream { 
             settings.write(it)
@@ -77,9 +110,9 @@ class OptionsController {
         model.font = text
         uiSettings.font = text
         
-        boolean showMonitor = view.monitorCheckbox.model.isSelected()
-        model.showMonitor = showMonitor
-        uiSettings.showMonitor = showMonitor
+//        boolean showMonitor = view.monitorCheckbox.model.isSelected()
+//        model.showMonitor = showMonitor
+//        uiSettings.showMonitor = showMonitor
         
         boolean clearCancelledDownloads = view.clearCancelledDownloadsCheckbox.model.isSelected()
         model.clearCancelledDownloads = clearCancelledDownloads
@@ -93,9 +126,9 @@ class OptionsController {
         model.excludeLocalResult = excludeLocalResult
         uiSettings.excludeLocalResult = excludeLocalResult
         
-        boolean showSearchHashes = view.showSearchHashesCheckbox.model.isSelected()
-        model.showSearchHashes = showSearchHashes
-        uiSettings.showSearchHashes = showSearchHashes
+//        boolean showSearchHashes = view.showSearchHashesCheckbox.model.isSelected()
+//        model.showSearchHashes = showSearchHashes
+//        uiSettings.showSearchHashes = showSearchHashes
         
         File uiSettingsFile = new File(core.home, "gui.properties")
         uiSettingsFile.withOutputStream { 
@@ -110,4 +143,15 @@ class OptionsController {
         view.d.setVisible(false)
         mvcGroup.destroy()
     }
+    
+    @ControllerAction
+    void downloadLocation() {
+        def chooser = new JFileChooser()
+        chooser.setFileHidingEnabled(false)
+        chooser.setDialogTitle("Select location for downloaded files")
+        chooser.setFileSelectionMode(JFileChooser.DIRECTORIES_ONLY)
+        int rv = chooser.showOpenDialog(null)
+        if (rv == JFileChooser.APPROVE_OPTION)
+            model.downloadLocation = chooser.getSelectedFile().getAbsolutePath()
+    }
 }

gui/griffon-app/lifecycle/Initialize.groovy

@@ -43,6 +43,8 @@ class Initialize extends AbstractLifecycleHandler {
         
         application.context.put("muwire-home", home.getAbsolutePath())
         
+        System.getProperties().setProperty("awt.useSystemAAFontSettings", "true")
+        
         def guiPropsFile = new File(home, "gui.properties")
         UISettings uiSettings
         if (guiPropsFile.exists()) {

gui/griffon-app/lifecycle/Ready.groovy

@@ -48,6 +48,8 @@ class Ready extends AbstractLifecycleHandler {
         } else {
             log.info("creating new properties")
             props = new MuWireSettings()
+            props.embeddedRouter = Boolean.parseBoolean(System.getProperties().getProperty("embeddedRouter"))
+            props.updateType = System.getProperty("updateType","jar")
             def nickname
             while (true) {
                 nickname = JOptionPane.showInputDialog(null,
@@ -74,6 +76,7 @@ class Ready extends AbstractLifecycleHandler {
                 props.downloadLocation = new File(portableDownloads)
             } else {
                 def chooser = new JFileChooser()
+                chooser.setFileHidingEnabled(false)
                 chooser.setDialogTitle("Select a directory where downloads will be saved")
                 chooser.setFileSelectionMode(JFileChooser.DIRECTORIES_ONLY)
                 int rv = chooser.showOpenDialog(null)
@@ -98,6 +101,9 @@ class Ready extends AbstractLifecycleHandler {
                 "Can't connect to I2P router", JOptionPane.WARNING_MESSAGE)
             System.exit(0)
         }
+        Runtime.getRuntime().addShutdownHook({
+            core.shutdown() 
+        })
         core.startServices()
         application.context.put("muwire-settings", props)
         application.context.put("core",core)

gui/griffon-app/models/com/muwire/gui/I2PStatusModel.groovy

@@ -0,0 +1,28 @@
+package com.muwire.gui
+
+import javax.annotation.Nonnull
+
+import griffon.core.artifact.GriffonModel
+import griffon.inject.MVCMember
+import griffon.transform.Observable
+import griffon.metadata.ArtifactProviderFor
+
+@ArtifactProviderFor(GriffonModel)
+class I2PStatusModel {
+    @MVCMember @Nonnull
+    I2PStatusController controller
+    
+    @Observable int ntcpConnections
+    @Observable int ssuConnections
+    @Observable String networkStatus
+    @Observable boolean floodfill
+    @Observable int participatingTunnels
+    @Observable int activePeers
+    @Observable int receiveBps
+    @Observable int sendBps
+    @Observable int participatingBW
+    
+    void mvcGroupInit(Map<String,String> args) {
+        controller.refresh()
+    }
+}

gui/griffon-app/models/com/muwire/gui/MainFrameModel.groovy

@@ -11,11 +11,13 @@ import com.muwire.core.Core
 import com.muwire.core.InfoHash
 import com.muwire.core.MuWireSettings
 import com.muwire.core.Persona
+import com.muwire.core.RouterDisconnectedEvent
 import com.muwire.core.connection.ConnectionAttemptStatus
 import com.muwire.core.connection.ConnectionEvent
 import com.muwire.core.connection.DisconnectionEvent
 import com.muwire.core.download.DownloadStartedEvent
 import com.muwire.core.download.Downloader
+import com.muwire.core.files.AllFilesLoadedEvent
 import com.muwire.core.files.FileDownloadedEvent
 import com.muwire.core.files.FileHashedEvent
 import com.muwire.core.files.FileLoadedEvent
@@ -27,6 +29,7 @@ import com.muwire.core.search.UIResultEvent
 import com.muwire.core.trust.TrustEvent
 import com.muwire.core.trust.TrustService
 import com.muwire.core.update.UpdateAvailableEvent
+import com.muwire.core.update.UpdateDownloadedEvent
 import com.muwire.core.upload.UploadEvent
 import com.muwire.core.upload.UploadFinishedEvent
 
@@ -50,6 +53,7 @@ class MainFrameModel {
     MainFrameController controller
     @Inject @Nonnull GriffonApplication application
     @Observable boolean coreInitialized = false
+    @Observable boolean routerPresent
     
     def results = new ConcurrentHashMap<>()
     def downloads = []
@@ -67,12 +71,14 @@ class MainFrameModel {
     @Observable boolean trustButtonsEnabled
     @Observable boolean cancelButtonEnabled
     @Observable boolean retryButtonEnabled
+    @Observable boolean pauseButtonEnabled
+    @Observable String resumeButtonText
     
     private final Set<InfoHash> infoHashes = new HashSet<>()
     
     private final Set<InfoHash> downloadInfoHashes = new HashSet<>()
 
-    volatile Core core    
+    @Observable volatile Core core    
 
     private long lastRetryTime = System.currentTimeMillis()
     
@@ -120,6 +126,7 @@ class MainFrameModel {
         application.addPropertyChangeListener("core", {e ->
             coreInitialized = (e.getNewValue() != null)
             core = e.getNewValue()
+            routerPresent = core.router != null
             me = core.me.getHumanReadableName()
             core.eventBus.register(UIResultEvent.class, this)
             core.eventBus.register(UIResultBatchEvent.class, this)
@@ -135,8 +142,13 @@ class MainFrameModel {
             core.eventBus.register(UpdateAvailableEvent.class, this)
             core.eventBus.register(FileDownloadedEvent.class, this)
             core.eventBus.register(FileUnsharedEvent.class, this)
+            core.eventBus.register(RouterDisconnectedEvent.class, this)
+            core.eventBus.register(AllFilesLoadedEvent.class, this)
+            core.eventBus.register(UpdateDownloadedEvent.class, this)
             
             timer.schedule({
+                if (core.shutdown.get())
+                    return
                 int retryInterval = core.muOptions.downloadRetryInterval
                 if (retryInterval > 0) {
                     retryInterval *= 60000
@@ -161,14 +173,28 @@ class MainFrameModel {
                 trusted.addAll(core.trustService.good.values())
                 distrusted.addAll(core.trustService.bad.values())
                 
-                watched.addAll(core.muOptions.watchedDirectories)
-                builder.getVariable("watched-directories-table").model.fireTableDataChanged()
-                watched.each { core.eventBus.publish(new FileSharedEvent(file : new File(it))) }
+                
+                resumeButtonText = "Retry"
             }
         })
         
     }
     
+    void onAllFilesLoadedEvent(AllFilesLoadedEvent e) {
+        runInsideUIAsync {
+            watched.addAll(core.muOptions.watchedDirectories)
+            builder.getVariable("watched-directories-table").model.fireTableDataChanged()
+            watched.each { core.eventBus.publish(new FileSharedEvent(file : new File(it))) }
+        }
+    }
+    
+    void onUpdateDownloadedEvent(UpdateDownloadedEvent e) {
+        runInsideUIAsync {
+            JOptionPane.showMessageDialog(null, "MuWire $e.version has been downloaded.  You can update now", 
+                "Update Downloaded", JOptionPane.INFORMATION_MESSAGE)
+        }
+    }
+    
     void onUIResultEvent(UIResultEvent e) {
         MVCGroup resultsGroup = results.get(e.uuid)
         resultsGroup?.model.handleResult(e)
@@ -176,7 +202,7 @@ class MainFrameModel {
     
     void onUIResultBatchEvent(UIResultBatchEvent e) {
         MVCGroup resultsGroup = results.get(e.uuid)
-        resultsGroup?.model.handleResultBatch(e.results)
+        resultsGroup?.model?.handleResultBatch(e.results)
     }
     
     void onDownloadStartedEvent(DownloadStartedEvent e) {
@@ -282,8 +308,10 @@ class MainFrameModel {
             updateTablePreservingSelection("trusted-table")
             updateTablePreservingSelection("distrusted-table")
             
-            results.values().each { 
-                it.view.pane.getClientProperty("results-table")?.model.fireTableDataChanged()
+            results.values().each { MVCGroup group -> 
+                if (group.alive) {
+                    group.view.pane.getClientProperty("results-table")?.model.fireTableDataChanged()
+                }
             }
         }
     }
@@ -335,6 +363,14 @@ class MainFrameModel {
         }
     }
     
+    void onRouterDisconnectedEvent(RouterDisconnectedEvent e) {
+        runInsideUIAsync {
+            JOptionPane.showMessageDialog(null, "MuWire lost connection to the I2P router and will now exit.",
+                "Connection to I2P router lost", JOptionPane.WARNING_MESSAGE)
+            System.exit(0)
+        }
+    }
+    
     void onFileDownloadedEvent(FileDownloadedEvent e) {
         if (!core.muOptions.shareDownloadedFiles)
             return

gui/griffon-app/models/com/muwire/gui/MuWireStatusModel.groovy

@@ -0,0 +1,25 @@
+package com.muwire.gui
+
+import javax.annotation.Nonnull
+
+import griffon.core.artifact.GriffonModel
+import griffon.inject.MVCMember
+import griffon.transform.Observable
+import griffon.metadata.ArtifactProviderFor
+
+@ArtifactProviderFor(GriffonModel)
+class MuWireStatusModel {
+    
+    @MVCMember @Nonnull
+    MuWireStatusController controller
+    
+    @Observable int incomingConnections
+    @Observable int outgoingConnections
+    @Observable int knownHosts
+    @Observable int sharedFiles
+    @Observable int downloads
+    
+    void mvcGroupInit(Map<String,String> args) {
+        controller.refresh()
+    }
+}

gui/griffon-app/models/com/muwire/gui/OptionsModel.groovy

@@ -11,14 +11,18 @@ import griffon.metadata.ArtifactProviderFor
 class OptionsModel {
     @Observable String downloadRetryInterval 
     @Observable String updateCheckInterval
+    @Observable boolean autoDownloadUpdate
     @Observable boolean onlyTrusted
     @Observable boolean shareDownloadedFiles
+    @Observable String downloadLocation
     
     // i2p options
     @Observable String inboundLength
     @Observable String inboundQuantity
     @Observable String outboundLength
     @Observable String outboundQuantity
+    @Observable String i2pUDPPort
+    @Observable String i2pNTCPPort
     
     // gui options
     @Observable boolean showMonitor
@@ -29,18 +33,26 @@ class OptionsModel {
     @Observable boolean excludeLocalResult
     @Observable boolean showSearchHashes
     
+    // bw options
+    @Observable String inBw
+    @Observable String outBw
+    
     void mvcGroupInit(Map<String, String> args) {
         MuWireSettings settings = application.context.get("muwire-settings")
         downloadRetryInterval = settings.downloadRetryInterval
         updateCheckInterval = settings.updateCheckInterval
+        autoDownloadUpdate = settings.autoDownloadUpdate
         onlyTrusted = !settings.allowUntrusted()
         shareDownloadedFiles = settings.shareDownloadedFiles
+        downloadLocation = settings.downloadLocation.getAbsolutePath()
         
         Core core = application.context.get("core")
         inboundLength = core.i2pOptions["inbound.length"]
         inboundQuantity = core.i2pOptions["inbound.quantity"]
         outboundLength = core.i2pOptions["outbound.length"]
         outboundQuantity = core.i2pOptions["outbound.quantity"]
+        i2pUDPPort = core.i2pOptions["i2np.udp.port"]
+        i2pNTCPPort = core.i2pOptions["i2np.ntcp.port"]
         
         UISettings uiSettings = application.context.get("ui-settings")
         showMonitor = uiSettings.showMonitor
@@ -50,5 +62,10 @@ class OptionsModel {
         clearFinishedDownloads = uiSettings.clearFinishedDownloads
         excludeLocalResult = uiSettings.excludeLocalResult
         showSearchHashes = uiSettings.showSearchHashes
+        
+        if (core.router != null) {
+            inBw = String.valueOf(settings.inBw)
+            outBw = String.valueOf(settings.outBw)
+        }
     }
 }

gui/griffon-app/views/com/muwire/gui/EventListView.groovy

@@ -21,10 +21,10 @@ class EventListView {
             application(size: [320, 80], id: 'event-list',
                 locationRelativeTo : null,
                 title: application.configuration['application.title'],
-                iconImage:   imageIcon('/griffon-icon-48x48.png').image,
-                iconImages: [imageIcon('/griffon-icon-48x48.png').image,
-                             imageIcon('/griffon-icon-32x32.png').image,
-                             imageIcon('/griffon-icon-16x16.png').image],
+                iconImage:   imageIcon('/MuWire-48x48.png').image,
+                iconImages: [imageIcon('/MuWire-48x48.png').image,
+                             imageIcon('/MuWire-32x32.png').image,
+                             imageIcon('/MuWire-16x16.png').image],
                  visible: bind { !model.coreInitialized} ) {
                 panel {
                     vbox {

gui/griffon-app/views/com/muwire/gui/I2PStatusView.groovy

@@ -0,0 +1,80 @@
+package com.muwire.gui
+
+import griffon.core.artifact.GriffonView
+import griffon.inject.MVCMember
+import griffon.metadata.ArtifactProviderFor
+
+import javax.swing.JDialog
+import javax.swing.JPanel
+import javax.swing.SwingConstants
+
+import java.awt.BorderLayout
+import java.awt.event.WindowAdapter
+import java.awt.event.WindowEvent
+
+import javax.annotation.Nonnull
+
+@ArtifactProviderFor(GriffonView)
+class I2PStatusView {
+    @MVCMember @Nonnull
+    FactoryBuilderSupport builder
+    @MVCMember @Nonnull
+    I2PStatusModel model
+
+    def mainFrame
+    def dialog
+    def panel
+    def buttonsPanel
+    
+    void initUI() {
+        mainFrame = application.windowManager.findWindow("main-frame")
+        
+        dialog = new JDialog(mainFrame, "I2P Status", true)
+        
+        panel = builder.panel {
+            gridBagLayout()
+            label(text : "Network status", constraints : gbc(gridx:0, gridy:0))
+            label(text : bind {model.networkStatus}, constraints : gbc(gridx: 1, gridy:0))
+            label(text: "Floodfill", constraints : gbc(gridx: 0, gridy : 1))
+            label(text : bind {model.floodfill}, constraints : gbc(gridx:1, gridy:1))
+            label(text : "NTCP Connections", constraints : gbc(gridx:0, gridy:2))
+            label(text : bind {model.ntcpConnections}, constraints : gbc(gridx: 1, gridy:2))
+            label(text : "SSU Connections", constraints : gbc(gridx:0, gridy:3))
+            label(text : bind {model.ssuConnections}, constraints : gbc(gridx: 1, gridy:3))
+            label(text : "Participating Tunnels", constraints : gbc(gridx:0, gridy:4))
+            label(text : bind {model.participatingTunnels}, constraints : gbc(gridx: 1, gridy:4))
+            label(text : "Participating Bandwidth", constraints : gbc(gridx:0, gridy:5))
+            label(text : bind {model.participatingBW}, constraints : gbc(gridx: 1, gridy:6))
+            label(text : "Active Peers", constraints : gbc(gridx:0, gridy:6))
+            label(text : bind {model.activePeers}, constraints : gbc(gridx: 1, gridy:6))
+            label(text : "Receive Bps (15 seconds)", constraints : gbc(gridx:0, gridy:7))
+            label(text : bind {model.receiveBps}, constraints : gbc(gridx: 1, gridy:7))
+            label(text : "Send Bps (15 seconds)", constraints : gbc(gridx:0, gridy:8))
+            label(text : bind {model.sendBps}, constraints : gbc(gridx: 1, gridy:8))
+        }
+        
+        buttonsPanel = builder.panel {
+            gridBagLayout()
+            button(text : "Refresh", constraints: gbc(gridx: 0, gridy: 0), refreshAction)
+            button(text : "Close", constraints : gbc(gridx : 1, gridy :0), closeAction)
+        }
+    }
+    
+    void mvcGroupInit(Map<String,String> args) {
+        JPanel statusPanel = new JPanel()
+        statusPanel.setLayout(new BorderLayout())
+        statusPanel.add(panel, BorderLayout.CENTER)
+        statusPanel.add(buttonsPanel, BorderLayout.SOUTH)
+        
+        dialog.getContentPane().add(statusPanel)
+        dialog.pack()
+        dialog.setLocationRelativeTo(mainFrame)
+        dialog.setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE)
+        dialog.addWindowListener(new WindowAdapter() {
+            public void windowClosed(WindowEvent e) {
+                mvcGroup.destroy()
+            }
+        })
+        dialog.show()
+    }
+}

gui/griffon-app/views/com/muwire/gui/MainFrameView.groovy

@@ -22,6 +22,7 @@ import javax.swing.border.Border
 import javax.swing.table.DefaultTableCellRenderer
 
 import com.muwire.core.Constants
+import com.muwire.core.MuWireSettings
 import com.muwire.core.download.Downloader
 import com.muwire.core.files.FileSharedEvent
 
@@ -52,23 +53,30 @@ class MainFrameView {
     def downloadsTable
     def lastDownloadSortEvent
     def lastSharedSortEvent
+    def lastWatchedSortEvent
     
     void initUI() {
         UISettings settings = application.context.get("ui-settings")
         builder.with {
             application(size : [1024,768], id: 'main-frame',
             locationRelativeTo : null,
-            title: application.configuration['application.title'] + " " + metadata["application.version"],
-            iconImage:   imageIcon('/griffon-icon-48x48.png').image,
-            iconImages: [imageIcon('/griffon-icon-48x48.png').image,
-                imageIcon('/griffon-icon-32x32.png').image,
-                imageIcon('/griffon-icon-16x16.png').image],
+            title: application.configuration['application.title'] + " " + 
+                metadata["application.version"] + " revision " + metadata["build.revision"],
+            iconImage:   imageIcon('/MuWire-48x48.png').image,
+            iconImages: [imageIcon('/MuWire-48x48.png').image,
+                imageIcon('/MuWire-32x32.png').image,
+                imageIcon('/MuWire-16x16.png').image],
             pack : false,
             visible : bind { model.coreInitialized }) {
                 menuBar {
                     menu (text : "Options") {
                         menuItem("Configuration", actionPerformed : {mvcGroup.createMVCGroup("Options")})
                     }
+                    menu (text : "Status") {
+                        menuItem("MuWire", actionPerformed : {mvcGroup.createMVCGroup("mu-wire-status")})
+                        MuWireSettings muSettings = application.context.get("muwire-settings")
+                        menuItem("I2P", enabled : bind {model.routerPresent}, actionPerformed: {mvcGroup.createMVCGroup("i-2-p-status")})
+                    }
                 }
                 borderLayout()
                 panel (border: etchedBorder(), constraints : BorderLayout.NORTH) {
@@ -134,8 +142,9 @@ class MainFrameView {
                                     }
                                 }
                                 panel (constraints : BorderLayout.SOUTH) {
+                                    button(text: "Pause", enabled : bind {model.pauseButtonEnabled}, pauseAction)
                                     button(text: "Cancel", enabled : bind {model.cancelButtonEnabled }, cancelAction )
-                                    button(text: "Retry", enabled : bind {model.retryButtonEnabled}, resumeAction)
+                                    button(text: bind { model.resumeButtonText }, enabled : bind {model.retryButtonEnabled}, resumeAction)
                                 }
                             }
                         }
@@ -165,8 +174,8 @@ class MainFrameView {
                                 scrollPane(constraints : BorderLayout.CENTER) {
                                     table(id : "shared-files-table", autoCreateRowSorter: true) {
                                         tableModel(list : model.shared) {
-                                            closureColumn(header : "Name", preferredWidth : 500, type : String, read : {row -> row.file.getAbsolutePath()})
-                                            closureColumn(header : "Size", preferredWidth : 100, type : Long, read : {row -> row.file.length() })
+                                            closureColumn(header : "Name", preferredWidth : 500, type : String, read : {row -> row.getCachedPath()})
+                                            closureColumn(header : "Size", preferredWidth : 100, type : Long, read : {row -> row.getCachedLength() })
                                         }
                                     }
                                 }
@@ -183,11 +192,14 @@ class MainFrameView {
                                         closureColumn(header : "Name", type : String, read : {row -> row.getName() })
                                         closureColumn(header : "Progress", type : String, read : { row ->
                                             int percent = row.getProgress()
-                                            "$percent%"
+                                            "$percent% of piece".toString()
                                         })
                                         closureColumn(header : "Downloader", type : String, read : { row -> 
                                             row.getDownloader()
                                         })
+                                        closureColumn(header : "Remote Pieces", type : String, read : { row ->
+                                            "${row.getDonePieces()}/${row.getTotalPieces()}".toString()
+                                        })
                                     }
                                 }
                             }
@@ -294,6 +306,7 @@ class MainFrameView {
             if (selectedRow < 0) {
                 model.cancelButtonEnabled = false
                 model.retryButtonEnabled = false
+                model.pauseButtonEnabled = false
                 return
             }
             def downloader = model.downloads[selectedRow]?.downloader
@@ -304,15 +317,25 @@ class MainFrameView {
                 case Downloader.DownloadState.DOWNLOADING :
                 case Downloader.DownloadState.HASHLIST:
                 model.cancelButtonEnabled = true
+                model.pauseButtonEnabled = true
                 model.retryButtonEnabled = false
                 break
                 case Downloader.DownloadState.FAILED:
                 model.cancelButtonEnabled = true
                 model.retryButtonEnabled = true
+                model.resumeButtonText = "Retry"
+                model.pauseButtonEnabled = false
+                break
+                case Downloader.DownloadState.PAUSED:
+                model.cancelButtonEnabled = true
+                model.retryButtonEnabled = true
+                model.resumeButtonText = "Resume"
+                model.pauseButtonEnabled = false
                 break
                 default:
                 model.cancelButtonEnabled = false
                 model.retryButtonEnabled = false
+                model.pauseButtonEnabled = false
             }
         })
         
@@ -344,11 +367,12 @@ class MainFrameView {
         sharedFilesTable.rowSorter.setSortsOnUpdates(true)
         
         JPopupMenu sharedFilesMenu = new JPopupMenu()
-//        JMenuItem unshareSelectedFiles = new JMenuItem("Unshare selected files")
-//        unshareSelectedFiles.addActionListener({mvcGroup.controller.unshareSelectedFiles()})
         JMenuItem copyHashToClipboard = new JMenuItem("Copy hash to clipboard")
-        copyHashToClipboard.addActionListener({mvcGroup.view.copyHashToClipboard(sharedFilesTable)})
+        copyHashToClipboard.addActionListener({mvcGroup.view.copyHashToClipboard()})
         sharedFilesMenu.add(copyHashToClipboard)
+        JMenuItem unshareSelectedFiles = new JMenuItem("Unshare selected files")
+        unshareSelectedFiles.addActionListener({mvcGroup.controller.unshareSelectedFile()})
+        sharedFilesMenu.add(unshareSelectedFiles)
         sharedFilesTable.addMouseListener(new MouseAdapter() {
             @Override
             public void mouseReleased(MouseEvent e) {
@@ -381,19 +405,48 @@ class MainFrameView {
             }
         })
         
+        // watched directories table
+        def watchedTable = builder.getVariable("watched-directories-table")
+        watchedTable.rowSorter.addRowSorterListener({evt -> lastWatchedSortEvent = evt})
+        watchedTable.rowSorter.setSortsOnUpdates(true)
+        JPopupMenu watchedMenu = new JPopupMenu()
+        JMenuItem stopWatching = new JMenuItem("Stop sharing")
+        stopWatching.addActionListener({mvcGroup.controller.stopWatchingDirectory()})
+        watchedMenu.add(stopWatching)
+        watchedTable.addMouseListener(new MouseAdapter() {
+            @Override
+            public void mouseReleased(MouseEvent e) {
+                if (e.isPopupTrigger())
+                    showPopupMenu(watchedMenu, e)
+            }
+            @Override
+            public void mousePressed(MouseEvent e) {
+                if (e.isPopupTrigger())
+                    showPopupMenu(watchedMenu, e)
+            }
+        })
+        
     }
     
     private static void showPopupMenu(JPopupMenu menu, MouseEvent event) {
         menu.show(event.getComponent(), event.getX(), event.getY())
     }
     
-    def copyHashToClipboard(JTable sharedFilesTable) {
+    def selectedSharedFile() {
+        def sharedFilesTable = builder.getVariable("shared-files-table")
         int selected = sharedFilesTable.getSelectedRow()
         if (selected < 0)
-            return
-        if (lastSharedSortEvent != null) 
+            return null
+        if (lastSharedSortEvent != null)
             selected = sharedFilesTable.rowSorter.convertRowIndexToModel(selected)
-        String root = Base64.encode(model.shared[selected].infoHash.getRoot())
+        model.shared[selected]
+    }
+    
+    def copyHashToClipboard() {
+        def selected = selectedSharedFile()
+        if (selected == null)
+            return
+        String root = Base64.encode(selected.infoHash.getRoot())
         StringSelection selection = new StringSelection(root)
         def clipboard = Toolkit.getDefaultToolkit().getSystemClipboard()
         clipboard.setContents(selection, null)
@@ -420,21 +473,32 @@ class MainFrameView {
         int selected = selectedDownloaderRow()
         if (selected < 0)
             return
+        boolean pauseEnabled = false
         boolean cancelEnabled = false
         boolean retryEnabled = false
+        String resumeText = "Retry"
         Downloader downloader = model.downloads[selected].downloader
         switch(downloader.currentState) {
             case Downloader.DownloadState.DOWNLOADING:
             case Downloader.DownloadState.HASHLIST:
             case Downloader.DownloadState.CONNECTING:
+                pauseEnabled = true
                 cancelEnabled = true
                 retryEnabled = false
                 break
             case Downloader.DownloadState.FAILED:
+                pauseEnabled = false
                 cancelEnabled = true
                 retryEnabled = true
                 break
+            case Downloader.DownloadState.PAUSED:
+                pauseEnabled = false
+                cancelEnabled = true
+                retryEnabled = true
+                resumeText = "Resume"
+                break
             default :
+                pauseEnabled = false
                 cancelEnabled = false
                 retryEnabled = false
         }
@@ -449,6 +513,12 @@ class MainFrameView {
         })
         menu.add(copyHashToClipboard)
         
+        if (pauseEnabled) {
+            JMenuItem pause = new JMenuItem("Pause")
+            pause.addActionListener({mvcGroup.controller.pause()})
+            menu.add(pause)
+        }
+        
         if (cancelEnabled) {
             JMenuItem cancel = new JMenuItem("Cancel")
             cancel.addActionListener({mvcGroup.controller.cancel()})
@@ -456,7 +526,7 @@ class MainFrameView {
         }
         
         if (retryEnabled) {
-            JMenuItem retry = new JMenuItem("Retry")
+            JMenuItem retry = new JMenuItem(resumeText)
             retry.addActionListener({mvcGroup.controller.resume()})
             menu.add(retry)
         }
@@ -486,6 +556,7 @@ class MainFrameView {
     
     def shareFiles = {
         def chooser = new JFileChooser()
+        chooser.setFileHidingEnabled(false)
         chooser.setDialogTitle("Select file to share")
         chooser.setFileSelectionMode(JFileChooser.FILES_ONLY)
         int rv = chooser.showOpenDialog(null)
@@ -496,6 +567,7 @@ class MainFrameView {
     
     def watchDirectories = {
         def chooser = new JFileChooser()
+        chooser.setFileHidingEnabled(false)
         chooser.setDialogTitle("Select directory to watch")
         chooser.setFileSelectionMode(JFileChooser.DIRECTORIES_ONLY)
         int rv = chooser.showOpenDialog(null)
@@ -508,4 +580,14 @@ class MainFrameView {
             model.core.eventBus.publish(new FileSharedEvent(file : f))
         }
     }
+    
+    String getSelectedWatchedDirectory() {
+        def watchedTable = builder.getVariable("watched-directories-table")
+        int selectedRow = watchedTable.getSelectedRow()
+        if (selectedRow < 0)
+            return null
+        if (lastWatchedSortEvent != null)
+            selectedRow = watchedTable.rowSorter.convertRowIndexToModel(selectedRow)
+        model.watched[selectedRow]
+    }
 }

gui/griffon-app/views/com/muwire/gui/MuWireStatusView.groovy

@@ -0,0 +1,73 @@
+package com.muwire.gui
+
+import griffon.core.artifact.GriffonView
+import griffon.inject.MVCMember
+import griffon.metadata.ArtifactProviderFor
+
+import javax.swing.JDialog
+import javax.swing.JPanel
+import javax.swing.SwingConstants
+
+import com.muwire.core.Core
+
+import java.awt.BorderLayout
+import java.awt.event.WindowAdapter
+import java.awt.event.WindowEvent
+
+import javax.annotation.Nonnull
+
+@ArtifactProviderFor(GriffonView)
+class MuWireStatusView {
+    @MVCMember @Nonnull
+    FactoryBuilderSupport builder
+    @MVCMember @Nonnull
+    MuWireStatusModel model
+
+    def mainFrame
+    def dialog
+    def panel
+    def buttonsPanel
+    
+    void initUI() {
+        mainFrame = application.windowManager.findWindow("main-frame")
+        
+        dialog = new JDialog(mainFrame, "MuWire Status", true)
+        
+        panel = builder.panel {
+            gridBagLayout()
+            label(text : "Incoming connections", constraints : gbc(gridx:0, gridy:0))
+            label(text : bind {model.incomingConnections}, constraints : gbc(gridx:1, gridy:0))
+            label(text : "Outgoing connections", constraints : gbc(gridx:0, gridy:1))
+            label(text : bind {model.outgoingConnections}, constraints : gbc(gridx:1, gridy:1))
+            label(text : "Known hosts", constraints : gbc(gridx:0, gridy:2))
+            label(text : bind {model.knownHosts}, constraints : gbc(gridx:1, gridy:2))
+            label(text : "Shared files", constraints : gbc(gridx:0, gridy:3))
+            label(text : bind {model.sharedFiles}, constraints : gbc(gridx:1, gridy:3))
+            label(text : "Downloads", constraints : gbc(gridx:0, gridy:4))
+            label(text : bind {model.downloads}, constraints : gbc(gridx:1, gridy:4))
+        }
+        buttonsPanel = builder.panel {
+            gridBagLayout()
+            button(text : "Refresh", constraints: gbc(gridx: 0, gridy: 0), refreshAction)
+            button(text : "Close", constraints : gbc(gridx : 1, gridy :0), closeAction)
+        }
+    }
+    
+    void mvcGroupInit(Map<String,String> args) {
+        JPanel statusPanel = new JPanel()
+        statusPanel.setLayout(new BorderLayout())
+        statusPanel.add(panel, BorderLayout.CENTER)
+        statusPanel.add(buttonsPanel, BorderLayout.SOUTH)
+        
+        dialog.getContentPane().add(statusPanel)
+        dialog.pack()
+        dialog.setLocationRelativeTo(mainFrame)
+        dialog.setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE)
+        dialog.addWindowListener(new WindowAdapter() {
+            public void windowClosed(WindowEvent e) {
+                mvcGroup.destroy()
+            }
+        })
+        dialog.show()
+    }
+}

gui/griffon-app/views/com/muwire/gui/OptionsView.groovy

@@ -9,6 +9,8 @@ import javax.swing.JPanel
 import javax.swing.JTabbedPane
 import javax.swing.SwingConstants
 
+import com.muwire.core.Core
+
 import java.awt.BorderLayout
 import java.awt.event.WindowAdapter
 import java.awt.event.WindowEvent
@@ -26,9 +28,11 @@ class OptionsView {
     def p
     def i
     def u
+    def bandwidth
     
     def retryField
     def updateField
+    def autoDownloadUpdateCheckbox
     def allowUntrustedCheckbox
     def shareDownloadedCheckbox
     
@@ -36,6 +40,8 @@ class OptionsView {
     def inboundQuantityField
     def outboundLengthField
     def outboundQuantityField
+    def i2pUDPPortField
+    def i2pNTCPPortField
 
     def lnfField
     def monitorCheckbox
@@ -45,6 +51,10 @@ class OptionsView {
     def excludeLocalResultCheckbox
     def showSearchHashesCheckbox
     
+    
+    def inBwField
+    def outBwField
+    
     def buttonsPanel    
     
     def mainFrame
@@ -62,12 +72,19 @@ class OptionsView {
             label(text : "Check for updates every", constraints : gbc(gridx : 0, gridy: 1))
             updateField = textField(text : bind {model.updateCheckInterval }, columns : 2, constraints : gbc(gridx : 1, gridy: 1))
             label(text : "hours", constraints : gbc(gridx: 2, gridy : 1))
-
-            label(text : "Allow only trusted connections", constraints : gbc(gridx: 0, gridy : 2))
-            allowUntrustedCheckbox = checkBox(selected : bind {model.onlyTrusted}, constraints : gbc(gridx: 1, gridy : 2))
             
-            label(text : "Share downloaded files", constraints : gbc(gridx : 0, gridy:3))
-            shareDownloadedCheckbox = checkBox(selected : bind {model.shareDownloadedFiles}, constraints : gbc(gridx :1, gridy:3))
+            label(text : "Download updates automatically", constraints: gbc(gridx :0, gridy : 2))
+            autoDownloadUpdateCheckbox = checkBox(selected : bind {model.autoDownloadUpdate}, constraints : gbc(gridx:1, gridy : 2))
+
+            label(text : "Allow only trusted connections", constraints : gbc(gridx: 0, gridy : 3))
+            allowUntrustedCheckbox = checkBox(selected : bind {model.onlyTrusted}, constraints : gbc(gridx: 1, gridy : 3))
+            
+            label(text : "Share downloaded files", constraints : gbc(gridx : 0, gridy:4))
+            shareDownloadedCheckbox = checkBox(selected : bind {model.shareDownloadedFiles}, constraints : gbc(gridx :1, gridy:4))
+            
+            label(text : "Save downloaded files to:", constraints: gbc(gridx:0, gridy:5))
+            button(text : "Choose", constraints : gbc(gridx : 1, gridy:5), downloadLocationAction)
+            label(text : bind {model.downloadLocation}, constraints: gbc(gridx:0, gridy:6, gridwidth:2))
                         
         }
         i = builder.panel {
@@ -81,6 +98,15 @@ class OptionsView {
             outboundLengthField = textField(text : bind {model.outboundLength}, columns : 2, constraints : gbc(gridx:1, gridy:3))
             label(text : "Outbound Quantity", constraints : gbc(gridx:0, gridy:4))
             outboundQuantityField = textField(text : bind {model.outboundQuantity}, columns : 2, constraints : gbc(gridx:1, gridy:4))
+            
+            Core core = application.context.get("core")
+            if (core.router != null) {
+                label(text : "TCP Port", constraints : gbc(gridx :0, gridy: 5))
+                i2pNTCPPortField = textField(text : bind {model.i2pNTCPPort}, columns : 4, constraints : gbc(gridx:1, gridy:5))
+                label(text : "UDP Port", constraints : gbc(gridx :0, gridy: 6))
+                i2pUDPPortField = textField(text : bind {model.i2pUDPPort}, columns : 4, constraints : gbc(gridx:1, gridy:6))
+            }
+            
         }
         u = builder.panel {
             gridBagLayout()
@@ -89,17 +115,27 @@ class OptionsView {
             lnfField = textField(text : bind {model.lnf}, columns : 4, constraints : gbc(gridx : 1, gridy : 1))
             label(text : "Font", constraints : gbc(gridx: 0, gridy : 2))
             fontField = textField(text : bind {model.font}, columns : 4, constraints : gbc(gridx : 1, gridy:2))
-            label(text : "Show Monitor", constraints : gbc(gridx :0, gridy: 3))
-            monitorCheckbox = checkBox(selected : bind {model.showMonitor}, constraints : gbc(gridx : 1, gridy: 3))
+//            label(text : "Show Monitor", constraints : gbc(gridx :0, gridy: 3))
+//            monitorCheckbox = checkBox(selected : bind {model.showMonitor}, constraints : gbc(gridx : 1, gridy: 3))
             label(text : "Clear Cancelled Downloads", constraints: gbc(gridx: 0, gridy:4))
             clearCancelledDownloadsCheckbox = checkBox(selected : bind {model.clearCancelledDownloads}, constraints : gbc(gridx : 1, gridy:4))
             label(text : "Clear Finished Downloads", constraints: gbc(gridx: 0, gridy:5))
             clearFinishedDownloadsCheckbox = checkBox(selected : bind {model.clearFinishedDownloads}, constraints : gbc(gridx : 1, gridy:5))
             label(text : "Exclude Local Files From Results", constraints: gbc(gridx:0, gridy:6))
             excludeLocalResultCheckbox = checkBox(selected : bind {model.excludeLocalResult}, constraints : gbc(gridx: 1, gridy : 6))
-            label(text : "Show Hash Searches In Monitor", constraints: gbc(gridx:0, gridy:7))
-            showSearchHashesCheckbox = checkBox(selected : bind {model.showSearchHashes}, constraints : gbc(gridx: 1, gridy: 7))
+//            label(text : "Show Hash Searches In Monitor", constraints: gbc(gridx:0, gridy:7))
+//            showSearchHashesCheckbox = checkBox(selected : bind {model.showSearchHashes}, constraints : gbc(gridx: 1, gridy: 7))
         }
+        bandwidth = builder.panel {
+            gridBagLayout()
+            label(text : "Changing these settings requires a restart", constraints : gbc(gridx : 0, gridy : 0, gridwidth: 2))
+            label(text : "Inbound bandwidth (KB)", constraints : gbc(gridx: 0, gridy : 1))
+            inBwField = textField(text : bind {model.inBw}, columns : 3, constraints : gbc(gridx : 1, gridy : 1))
+            label(text : "Outbound bandwidth (KB)", constraints : gbc(gridx: 0, gridy : 2))
+            outBwField = textField(text : bind {model.outBw}, columns : 3, constraints : gbc(gridx : 1, gridy : 2))
+        }
+       
+        
         buttonsPanel = builder.panel {
             gridBagLayout()
             button(text : "Save", constraints : gbc(gridx : 1, gridy: 2), saveAction)
@@ -112,6 +148,10 @@ class OptionsView {
         tabbedPane.addTab("MuWire", p)
         tabbedPane.addTab("I2P", i)
         tabbedPane.addTab("GUI", u)
+        Core core = application.context.get("core")
+        if (core.router != null) {
+            tabbedPane.addTab("Bandwidth", bandwidth)
+        }
                 
         JPanel panel = new JPanel()
         panel.setLayout(new BorderLayout())

gui/src/integration-test/groovy/com/muwire/gui/I2PStatusIntegrationTest.groovy

@@ -0,0 +1,25 @@
+package com.muwire.gui
+
+import griffon.core.test.GriffonFestRule
+import org.fest.swing.fixture.FrameFixture
+import org.junit.Rule
+import org.junit.Test
+
+import static org.junit.Assert.fail
+
+class I2PStatusIntegrationTest {
+    static {
+        System.setProperty('griffon.swing.edt.violations.check', 'true')
+        System.setProperty('griffon.swing.edt.hang.monitor', 'true')
+    }
+
+    @Rule
+    public final GriffonFestRule fest = new GriffonFestRule()
+
+    private FrameFixture window
+
+    @Test
+    void smokeTest() {
+        fail('Not implemented yet!')
+    }
+}

gui/src/integration-test/groovy/com/muwire/gui/MuWireStatusIntegrationTest.groovy

@@ -0,0 +1,25 @@
+package com.muwire.gui
+
+import griffon.core.test.GriffonFestRule
+import org.fest.swing.fixture.FrameFixture
+import org.junit.Rule
+import org.junit.Test
+
+import static org.junit.Assert.fail
+
+class MuWireStatusIntegrationTest {
+    static {
+        System.setProperty('griffon.swing.edt.violations.check', 'true')
+        System.setProperty('griffon.swing.edt.hang.monitor', 'true')
+    }
+
+    @Rule
+    public final GriffonFestRule fest = new GriffonFestRule()
+
+    private FrameFixture window
+
+    @Test
+    void smokeTest() {
+        fail('Not implemented yet!')
+    }
+}

gui/src/main/groovy/com/muwire/gui/UISettings.groovy

@@ -12,12 +12,12 @@ class UISettings {
     
     UISettings(Properties props) {
         lnf = props.getProperty("lnf", "system")
-        showMonitor = Boolean.parseBoolean(props.getProperty("showMonitor", "true"))
+        showMonitor = Boolean.parseBoolean(props.getProperty("showMonitor", "false"))
         font = props.getProperty("font",null)
-        clearCancelledDownloads = Boolean.parseBoolean(props.getProperty("clearCancelledDownloads","false"))
+        clearCancelledDownloads = Boolean.parseBoolean(props.getProperty("clearCancelledDownloads","true"))
         clearFinishedDownloads = Boolean.parseBoolean(props.getProperty("clearFinishedDownloads","false"))
-        excludeLocalResult = Boolean.parseBoolean(props.getProperty("excludeLocalResult","false"))
-        showSearchHashes = Boolean.parseBoolean(props.getProperty("showSearchHashes","false"))
+        excludeLocalResult = Boolean.parseBoolean(props.getProperty("excludeLocalResult","true"))
+        showSearchHashes = Boolean.parseBoolean(props.getProperty("showSearchHashes","true"))
     }
     
     void write(OutputStream out) throws IOException {

gui/src/test/groovy/com/muwire/gui/I2PStatusControllerTest.groovy

@@ -0,0 +1,21 @@
+package com.muwire.gui
+
+import griffon.core.test.GriffonUnitRule
+import griffon.core.test.TestFor
+import org.junit.Rule
+import org.junit.Test
+
+import static org.junit.Assert.fail
+
+@TestFor(I2PStatusController)
+class I2PStatusControllerTest {
+    private I2PStatusController controller
+
+    @Rule
+    public final GriffonUnitRule griffon = new GriffonUnitRule()
+
+    @Test
+    void smokeTest() {
+        fail('Not yet implemented!')
+    }
+}

gui/src/test/groovy/com/muwire/gui/MuWireStatusControllerTest.groovy

@@ -0,0 +1,21 @@
+package com.muwire.gui
+
+import griffon.core.test.GriffonUnitRule
+import griffon.core.test.TestFor
+import org.junit.Rule
+import org.junit.Test
+
+import static org.junit.Assert.fail
+
+@TestFor(MuWireStatusController)
+class MuWireStatusControllerTest {
+    private MuWireStatusController controller
+
+    @Rule
+    public final GriffonUnitRule griffon = new GriffonUnitRule()
+
+    @Test
+    void smokeTest() {
+        fail('Not yet implemented!')
+    }
+}

host-cache/src/main/groovy/com/muwire/hostcache/HostCache.groovy

@@ -11,6 +11,7 @@ import net.i2p.client.I2PSession
 import net.i2p.client.I2PSessionMuxedListener
 import net.i2p.client.datagram.I2PDatagramDissector
 import net.i2p.client.datagram.I2PDatagramMaker
+import net.i2p.crypto.SigType
 import net.i2p.util.SystemVersion
 import net.i2p.data.*
 
@@ -43,7 +44,7 @@ public class HostCache {
         def session
         if (!keyfile.exists()) {
             def os = new FileOutputStream(keyfile);
-            myDest = i2pClient.createDestination(os)
+            myDest = i2pClient.createDestination(os, SigType.EdDSA_SHA512_Ed25519)
             os.close()
             println "No key.dat file was found, so creating a new destination."
             println "This is the destination you want to give out for your new HostCache"

logging/1_logging.properties

@@ -0,0 +1,61 @@
+############################################################
+#  	Default Logging Configuration File
+#
+# You can use a different file by specifying a filename
+# with the java.util.logging.config.file system property.  
+# For example java -Djava.util.logging.config.file=myfile
+############################################################
+
+############################################################
+#  	Global properties
+############################################################
+
+# "handlers" specifies a comma separated list of log Handler 
+# classes.  These handlers will be installed during VM startup.
+# Note that these classes must be on the system classpath.
+# By default we only configure a ConsoleHandler, which will only
+# show messages at the INFO and above levels.
+handlers= java.util.logging.FileHandler
+
+# To also add the FileHandler, use the following line instead.
+#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
+
+# Default global logging level.
+# This specifies which kinds of events are logged across
+# all loggers.  For any given facility this global level
+# can be overriden by a facility specific level
+# Note that the ConsoleHandler also has a separate level
+# setting to limit messages printed to the console.
+.level= SEVERE
+
+############################################################
+# Handler specific properties.
+# Describes specific configuration info for Handlers.
+############################################################
+
+# default file output is in user's home directory.
+java.util.logging.FileHandler.pattern = MuWire.log
+java.util.logging.FileHandler.limit = 50000000
+java.util.logging.FileHandler.count = 1
+java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
+
+# Limit the message that are printed on the console to INFO and above.
+java.util.logging.ConsoleHandler.level = INFO
+java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
+
+# Example to customize the SimpleFormatter output format 
+# to print one-line log message like this:
+#     <level>: <log message> [<date/time>]
+#
+#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
+
+java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
+
+############################################################
+# Facility specific properties.
+# Provides extra control for each logger.
+############################################################
+
+# For example, set the com.xyz.foo logger to only log SEVERE
+# messages:
+com.xyz.foo.level = SEVERE

logging/2_logging.properties

@@ -0,0 +1,61 @@
+############################################################
+#  	Default Logging Configuration File
+#
+# You can use a different file by specifying a filename
+# with the java.util.logging.config.file system property.  
+# For example java -Djava.util.logging.config.file=myfile
+############################################################
+
+############################################################
+#  	Global properties
+############################################################
+
+# "handlers" specifies a comma separated list of log Handler 
+# classes.  These handlers will be installed during VM startup.
+# Note that these classes must be on the system classpath.
+# By default we only configure a ConsoleHandler, which will only
+# show messages at the INFO and above levels.
+handlers= java.util.logging.FileHandler
+
+# To also add the FileHandler, use the following line instead.
+#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
+
+# Default global logging level.
+# This specifies which kinds of events are logged across
+# all loggers.  For any given facility this global level
+# can be overriden by a facility specific level
+# Note that the ConsoleHandler also has a separate level
+# setting to limit messages printed to the console.
+.level= WARNING
+
+############################################################
+# Handler specific properties.
+# Describes specific configuration info for Handlers.
+############################################################
+
+# default file output is in user's home directory.
+java.util.logging.FileHandler.pattern = MuWire.log
+java.util.logging.FileHandler.limit = 50000000
+java.util.logging.FileHandler.count = 1
+java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
+
+# Limit the message that are printed on the console to INFO and above.
+java.util.logging.ConsoleHandler.level = INFO
+java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
+
+# Example to customize the SimpleFormatter output format 
+# to print one-line log message like this:
+#     <level>: <log message> [<date/time>]
+#
+#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
+
+java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
+
+############################################################
+# Facility specific properties.
+# Provides extra control for each logger.
+############################################################
+
+# For example, set the com.xyz.foo logger to only log SEVERE
+# messages:
+com.xyz.foo.level = SEVERE

logging/3_logging.properties

@@ -0,0 +1,61 @@
+############################################################
+#  	Default Logging Configuration File
+#
+# You can use a different file by specifying a filename
+# with the java.util.logging.config.file system property.  
+# For example java -Djava.util.logging.config.file=myfile
+############################################################
+
+############################################################
+#  	Global properties
+############################################################
+
+# "handlers" specifies a comma separated list of log Handler 
+# classes.  These handlers will be installed during VM startup.
+# Note that these classes must be on the system classpath.
+# By default we only configure a ConsoleHandler, which will only
+# show messages at the INFO and above levels.
+handlers= java.util.logging.FileHandler
+
+# To also add the FileHandler, use the following line instead.
+#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
+
+# Default global logging level.
+# This specifies which kinds of events are logged across
+# all loggers.  For any given facility this global level
+# can be overriden by a facility specific level
+# Note that the ConsoleHandler also has a separate level
+# setting to limit messages printed to the console.
+.level= INFO
+
+############################################################
+# Handler specific properties.
+# Describes specific configuration info for Handlers.
+############################################################
+
+# default file output is in user's home directory.
+java.util.logging.FileHandler.pattern = MuWire.log
+java.util.logging.FileHandler.limit = 50000000
+java.util.logging.FileHandler.count = 1
+java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
+
+# Limit the message that are printed on the console to INFO and above.
+java.util.logging.ConsoleHandler.level = INFO
+java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
+
+# Example to customize the SimpleFormatter output format 
+# to print one-line log message like this:
+#     <level>: <log message> [<date/time>]
+#
+#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
+
+java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
+
+############################################################
+# Facility specific properties.
+# Provides extra control for each logger.
+############################################################
+
+# For example, set the com.xyz.foo logger to only log SEVERE
+# messages:
+com.xyz.foo.level = SEVERE

logging/4_logging.properties

@@ -0,0 +1,64 @@
+############################################################
+#  	Default Logging Configuration File
+#
+# You can use a different file by specifying a filename
+# with the java.util.logging.config.file system property.  
+# For example java -Djava.util.logging.config.file=myfile
+############################################################
+
+############################################################
+#  	Global properties
+############################################################
+
+# "handlers" specifies a comma separated list of log Handler 
+# classes.  These handlers will be installed during VM startup.
+# Note that these classes must be on the system classpath.
+# By default we only configure a ConsoleHandler, which will only
+# show messages at the INFO and above levels.
+handlers= java.util.logging.FileHandler
+
+# To also add the FileHandler, use the following line instead.
+#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
+
+# Default global logging level.
+# This specifies which kinds of events are logged across
+# all loggers.  For any given facility this global level
+# can be overriden by a facility specific level
+# Note that the ConsoleHandler also has a separate level
+# setting to limit messages printed to the console.
+.level= INFO 
+
+############################################################
+# Handler specific properties.
+# Describes specific configuration info for Handlers.
+############################################################
+
+# default file output is in user's home directory.
+java.util.logging.FileHandler.pattern = MuWire.log
+java.util.logging.FileHandler.limit = 150000000
+java.util.logging.FileHandler.count = 1
+java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
+
+# Limit the message that are printed on the console to INFO and above.
+java.util.logging.ConsoleHandler.level = INFO
+java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
+
+# Example to customize the SimpleFormatter output format 
+# to print one-line log message like this:
+#     <level>: <log message> [<date/time>]
+#
+#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
+
+java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
+
+############################################################
+# Facility specific properties.
+# Provides extra control for each logger.
+############################################################
+
+# For example, set the com.xyz.foo logger to only log SEVERE
+# messages:
+com.xyz.foo.level = SEVERE
+com.muwire.core.level = FINE
+net.i2p.client.streaming.impl.level = FINE
+net.i2p.client.impl.level = FINE