Compare commits

..

56 Commits

Author SHA1 Message Date
Zlatin Balevsky
538eca9297 Release 0.3.6 2019-06-23 08:54:28 +01:00
Zlatin Balevsky
e73a23d4a4 fix space not showing 2019-06-23 08:44:51 +01:00
Zlatin Balevsky
76e41a0383 fix restoring paused downloads 2019-06-23 08:42:45 +01:00
Zlatin Balevsky
7045927666 hide monitor options from gui 2019-06-23 08:02:28 +01:00
Zlatin Balevsky
5fb3086b42 update faq 2019-06-23 07:52:01 +01:00
Zlatin Balevsky
2de18227c1 persist pause state 2019-06-23 07:48:49 +01:00
Zlatin Balevsky
bd12a1de3d pause/resume downloads 2019-06-23 06:59:52 +01:00
Zlatin Balevsky
a3a91050c8 update todo 2019-06-23 01:50:30 +01:00
Zlatin Balevsky
6c1cc28e49 shutdown if connection to I2P router is lost 2019-06-22 17:32:12 +01:00
Zlatin Balevsky
b6e5b54f05 do not show monitor by default 2019-06-22 14:51:26 +01:00
Zlatin Balevsky
a6e559ec67 change some defaults 2019-06-22 06:54:49 +01:00
Zlatin Balevsky
f11badb824 update todo 2019-06-21 22:43:46 +01:00
Zlatin Balevsky
44da44ff6f Release 0.3.5 2019-06-21 22:35:54 +01:00
Zlatin Balevsky
aae3fc29ca add logging.properties with various degree of noisiness 2019-06-21 22:28:57 +01:00
Zlatin Balevsky
c30aa19d8b Merge branch 'download-mesh' 2019-06-21 22:26:17 +01:00
Zlatin Balevsky
c79e8712d0 correctly determine if uploader has requested piece 2019-06-21 20:36:33 +01:00
Zlatin Balevsky
ed12d78a48 clear pieces on cancel 2019-06-21 17:22:55 +01:00
Zlatin Balevsky
d27872cc8b investigate StringIndexOutOfBounds 2019-06-21 16:29:52 +01:00
Zlatin Balevsky
f794c39760 personas not destinations 2019-06-21 16:15:35 +01:00
Zlatin Balevsky
2be9c425f7 compute which pieces are requested 2019-06-21 16:09:57 +01:00
Zlatin Balevsky
ab5fea9216 416 if piece not downloaded 2019-06-21 16:03:20 +01:00
Zlatin Balevsky
d1c8328080 do not send alts if there aren't any 2019-06-21 15:39:00 +01:00
Zlatin Balevsky
89e761f53b write personas on the wire part1 2019-06-21 15:26:18 +01:00
Zlatin Balevsky
40410eba63 fix constructor 2019-06-21 14:57:53 +01:00
Zlatin Balevsky
85466a8e80 fix npe 2019-06-21 14:45:14 +01:00
Zlatin Balevsky
c210af7870 source partial uploads from incompletes file 2019-06-21 14:39:20 +01:00
Zlatin Balevsky
38ff49d28f downloaders get pieces from mesh manager 2019-06-21 14:17:10 +01:00
Zlatin Balevsky
710f9f52a8 send X-Have and X-Alts from uploader 2019-06-21 13:58:21 +01:00
Zlatin Balevsky
1b6eda5a40 skeleton of mesh manager 2019-06-21 13:34:00 +01:00
Zlatin Balevsky
1ee9ccf098 parse X-Have on uploader side 2019-06-21 12:55:25 +01:00
Zlatin Balevsky
0f07562de3 pass new sources to active downloaders 2019-06-21 12:39:16 +01:00
Zlatin Balevsky
6eb1aa07f5 key downloaders by infohash 2019-06-21 12:29:32 +01:00
Zlatin Balevsky
05b02834af parse X-Alt 2019-06-21 12:25:04 +01:00
Zlatin Balevsky
56125f6df8 refactor X-Have decoding logic 2019-06-21 09:32:10 +01:00
Zlatin Balevsky
8f9996848b send X-Have from downloader too 2019-06-21 09:25:28 +01:00
Zlatin Balevsky
dd655ed60f test for re-requesting available pieces 2019-06-21 09:12:42 +01:00
Zlatin Balevsky
8923c6ff7d exclude local results by default 2019-06-21 08:15:20 +01:00
Zlatin Balevsky
807ab22f8e test parsing of X-Have 2019-06-21 06:43:48 +01:00
Zlatin Balevsky
a26ad229ee more tests 2019-06-21 05:56:42 +01:00
Zlatin Balevsky
5504dd2251 tighten conditions 2019-06-21 05:45:11 +01:00
Zlatin Balevsky
f9777d29f4 get existing tests to pass 2019-06-21 05:41:49 +01:00
Zlatin Balevsky
b23226e8c6 wip on parsing X-Have from uploader 2019-06-21 05:30:56 +01:00
Zlatin Balevsky
1249ad29e0 claim pieces from list of available pieces 2019-06-21 04:42:02 +01:00
Zlatin Balevsky
7bb5e5b632 Release 0.3.4 2019-06-20 21:07:50 +01:00
Zlatin Balevsky
b2e43f9765 update split pattern and add unit test 2019-06-20 21:06:39 +01:00
Zlatin Balevsky
2aa73c203a Release 0.3.3 2019-06-20 18:08:02 +01:00
Zlatin Balevsky
18d2b56563 fix indexing 2019-06-20 17:57:36 +01:00
Zlatin Balevsky
a455b4ad6e redirect exceptions in result sender to log 2019-06-20 17:22:59 +01:00
Zlatin Balevsky
761b683a81 Release 0.3.2 2019-06-20 16:04:46 +01:00
Zlatin Balevsky
1d41bcd825 prevent empty tokens in search index 2019-06-20 16:02:48 +01:00
Zlatin Balevsky
f1ac038b55 update split pattern 2019-06-20 15:47:00 +01:00
Zlatin Balevsky
396c636e42 prevent empty search terms 2019-06-20 15:29:27 +01:00
Zlatin Balevsky
e32c858e90 update README with quick FAQ 2019-06-20 14:18:37 +01:00
Zlatin Balevsky
821555f3f1 Release 0.3.1 2019-06-20 14:02:22 +01:00
Zlatin Balevsky
089ab4f0d9 do not retry downloads if core is shut(ting) down 2019-06-20 13:40:04 +01:00
Zlatin Balevsky
948b6292fe add shutdown hook to shutdown core on SIGTERM 2019-06-20 13:29:15 +01:00
40 changed files with 1067 additions and 142 deletions

View File

@@ -31,3 +31,26 @@ The first time you run MuWire it will ask you to select a nickname. This nickna
### Known bugs and limitations ### Known bugs and limitations
* Many UI features you would expect are not there yet * Many UI features you would expect are not there yet
### Quick FAQ
* why is MuWire slow ?
- too few sources you're downloading from
- you can increase the number of tunnels by using more tunnels via Options->I2P Inbound/Outbound Quantity
the default is 4 and you could raise up to as high as 16 ( Caution !!!!)
* my search is not returning (enough) results !
- search is keyword or hash based
- keywords and hash(es) are NOT regexed or wildcarded so they have to be complete
so searching for 'musi' will not return results with 'music' - you have to search for 'music'
- ALL keywords have to match
- only use space for keyword separation
- if you already have the file in question it is not displayed ( can be changed via Options )
* what's this right click -> 'Copy hash to clipboard' for ?
- if you have a specific file you wish to share or download you can use the hash as a unique identifier
to make sure you have exactly the right file.
- you can share this hash with others to ensure they are getting the right file

View File

@@ -4,10 +4,6 @@ Not in any particular order yet
### Big Items ### Big Items
##### Alternate Locations
This helps peers discover new sources for a file while the download is in progress. Also makes sharing of partial files possible.
##### Bloom Filters ##### Bloom Filters
This reduces query traffic by not sending last hop queries to peers that definitely do not have the file This reduces query traffic by not sending last hop queries to peers that definitely do not have the file
@@ -38,8 +34,8 @@ To enable parsing of metadata from known file types and the user editing it or a
### Small Items ### Small Items
* Detect if router is dead and show warning or exit
* Wrapper of some kind for in-place upgrades * Wrapper of some kind for in-place upgrades
* Download file sequentially * Download file sequentially
* Unsharing of files * Unsharing of files
* Multiple-selection download, Ctrl-A * Multiple-selection download, Ctrl-A
* Pause/Resume downloads

View File

@@ -35,7 +35,7 @@ class Cli {
Core core Core core
try { try {
core = new Core(props, home, "0.3.0") core = new Core(props, home, "0.3.6")
} catch (Exception bad) { } catch (Exception bad) {
bad.printStackTrace(System.out) bad.printStackTrace(System.out)
println "Failed to initialize core, exiting" println "Failed to initialize core, exiting"

View File

@@ -53,7 +53,7 @@ class CliDownloader {
Core core Core core
try { try {
core = new Core(props, home, "0.3.0") core = new Core(props, home, "0.3.6")
} catch (Exception bad) { } catch (Exception bad) {
bad.printStackTrace(System.out) bad.printStackTrace(System.out)
println "Failed to initialize core, exiting" println "Failed to initialize core, exiting"

View File

@@ -11,5 +11,5 @@ class Constants {
public static final float DOWNLOAD_SEQUENTIAL_RATIO = 0.8f public static final float DOWNLOAD_SEQUENTIAL_RATIO = 0.8f
public static final String SPLIT_PATTERN = "[\\.,_-]" public static final String SPLIT_PATTERN = "[\\+\\-,\\.:;\\(\\)=_/\\\\\\!\\\"\\\'\\\$%\\|]"
} }

View File

@@ -1,6 +1,7 @@
package com.muwire.core package com.muwire.core
import java.nio.charset.StandardCharsets import java.nio.charset.StandardCharsets
import java.util.concurrent.atomic.AtomicBoolean
import com.muwire.core.connection.ConnectionAcceptor import com.muwire.core.connection.ConnectionAcceptor
import com.muwire.core.connection.ConnectionEstablisher import com.muwire.core.connection.ConnectionEstablisher
@@ -12,8 +13,11 @@ import com.muwire.core.connection.I2PConnector
import com.muwire.core.connection.LeafConnectionManager import com.muwire.core.connection.LeafConnectionManager
import com.muwire.core.connection.UltrapeerConnectionManager import com.muwire.core.connection.UltrapeerConnectionManager
import com.muwire.core.download.DownloadManager import com.muwire.core.download.DownloadManager
import com.muwire.core.download.SourceDiscoveredEvent
import com.muwire.core.download.UIDownloadCancelledEvent import com.muwire.core.download.UIDownloadCancelledEvent
import com.muwire.core.download.UIDownloadEvent import com.muwire.core.download.UIDownloadEvent
import com.muwire.core.download.UIDownloadPausedEvent
import com.muwire.core.download.UIDownloadResumedEvent
import com.muwire.core.files.FileDownloadedEvent import com.muwire.core.files.FileDownloadedEvent
import com.muwire.core.files.FileHashedEvent import com.muwire.core.files.FileHashedEvent
import com.muwire.core.files.FileHasher import com.muwire.core.files.FileHasher
@@ -27,6 +31,7 @@ import com.muwire.core.files.DirectoryWatcher
import com.muwire.core.hostcache.CacheClient import com.muwire.core.hostcache.CacheClient
import com.muwire.core.hostcache.HostCache import com.muwire.core.hostcache.HostCache
import com.muwire.core.hostcache.HostDiscoveredEvent import com.muwire.core.hostcache.HostDiscoveredEvent
import com.muwire.core.mesh.MeshManager
import com.muwire.core.search.QueryEvent import com.muwire.core.search.QueryEvent
import com.muwire.core.search.ResultsEvent import com.muwire.core.search.ResultsEvent
import com.muwire.core.search.ResultsSender import com.muwire.core.search.ResultsSender
@@ -45,6 +50,7 @@ import net.i2p.client.I2PSession
import net.i2p.client.streaming.I2PSocketManager import net.i2p.client.streaming.I2PSocketManager
import net.i2p.client.streaming.I2PSocketManagerFactory import net.i2p.client.streaming.I2PSocketManagerFactory
import net.i2p.client.streaming.I2PSocketOptions import net.i2p.client.streaming.I2PSocketOptions
import net.i2p.client.streaming.I2PSocketManager.DisconnectListener
import net.i2p.crypto.DSAEngine import net.i2p.crypto.DSAEngine
import net.i2p.crypto.SigType import net.i2p.crypto.SigType
import net.i2p.data.Destination import net.i2p.data.Destination
@@ -73,6 +79,8 @@ public class Core {
private final DownloadManager downloadManager private final DownloadManager downloadManager
private final DirectoryWatcher directoryWatcher private final DirectoryWatcher directoryWatcher
final FileManager fileManager final FileManager fileManager
final AtomicBoolean shutdown = new AtomicBoolean()
public Core(MuWireSettings props, File home, String myVersion) { public Core(MuWireSettings props, File home, String myVersion) {
this.home = home this.home = home
@@ -119,6 +127,7 @@ public class Core {
} }
socketManager.getDefaultOptions().setReadTimeout(60000) socketManager.getDefaultOptions().setReadTimeout(60000)
socketManager.getDefaultOptions().setConnectTimeout(30000) socketManager.getDefaultOptions().setConnectTimeout(30000)
socketManager.addDisconnectListener({eventBus.publish(new RouterDisconnectedEvent())} as DisconnectListener)
i2pSession = socketManager.getSession() i2pSession = socketManager.getSession()
def destination = new Destination() def destination = new Destination()
@@ -162,6 +171,10 @@ public class Core {
eventBus.register(FileDownloadedEvent.class, fileManager) eventBus.register(FileDownloadedEvent.class, fileManager)
eventBus.register(FileUnsharedEvent.class, fileManager) eventBus.register(FileUnsharedEvent.class, fileManager)
eventBus.register(SearchEvent.class, fileManager) eventBus.register(SearchEvent.class, fileManager)
log.info("initializing mesh manager")
MeshManager meshManager = new MeshManager(fileManager)
eventBus.register(SourceDiscoveredEvent.class, meshManager)
log.info "initializing persistence service" log.info "initializing persistence service"
persisterService = new PersisterService(new File(home, "files.json"), eventBus, 15000, fileManager) persisterService = new PersisterService(new File(home, "files.json"), eventBus, 15000, fileManager)
@@ -200,14 +213,17 @@ public class Core {
eventBus.register(ResultsEvent.class, searchManager) eventBus.register(ResultsEvent.class, searchManager)
log.info("initializing download manager") log.info("initializing download manager")
downloadManager = new DownloadManager(eventBus, i2pConnector, home, me) downloadManager = new DownloadManager(eventBus, trustService, meshManager, props, i2pConnector, home, me)
eventBus.register(UIDownloadEvent.class, downloadManager) eventBus.register(UIDownloadEvent.class, downloadManager)
eventBus.register(UILoadedEvent.class, downloadManager) eventBus.register(UILoadedEvent.class, downloadManager)
eventBus.register(FileDownloadedEvent.class, downloadManager) eventBus.register(FileDownloadedEvent.class, downloadManager)
eventBus.register(UIDownloadCancelledEvent.class, downloadManager) eventBus.register(UIDownloadCancelledEvent.class, downloadManager)
eventBus.register(SourceDiscoveredEvent.class, downloadManager)
eventBus.register(UIDownloadPausedEvent.class, downloadManager)
eventBus.register(UIDownloadResumedEvent.class, downloadManager)
log.info("initializing upload manager") log.info("initializing upload manager")
UploadManager uploadManager = new UploadManager(eventBus, fileManager) UploadManager uploadManager = new UploadManager(eventBus, fileManager, meshManager, downloadManager)
log.info("initializing connection establisher") log.info("initializing connection establisher")
connectionEstablisher = new ConnectionEstablisher(eventBus, i2pConnector, props, connectionManager, hostCache) connectionEstablisher = new ConnectionEstablisher(eventBus, i2pConnector, props, connectionManager, hostCache)
@@ -241,6 +257,10 @@ public class Core {
} }
public void shutdown() { public void shutdown() {
if (!shutdown.compareAndSet(false, true)) {
log.info("already shutting down")
return
}
log.info("shutting down download manageer") log.info("shutting down download manageer")
downloadManager.shutdown() downloadManager.shutdown()
log.info("shutting down connection acceeptor") log.info("shutting down connection acceeptor")
@@ -277,7 +297,7 @@ public class Core {
} }
} }
Core core = new Core(props, home, "0.3.0") Core core = new Core(props, home, "0.3.6")
core.startServices() core.startServices()
// ... at the end, sleep or execute script // ... at the end, sleep or execute script

View File

@@ -30,8 +30,8 @@ class MuWireSettings {
nickname = props.getProperty("nickname","MuWireUser") nickname = props.getProperty("nickname","MuWireUser")
downloadLocation = new File((String)props.getProperty("downloadLocation", downloadLocation = new File((String)props.getProperty("downloadLocation",
System.getProperty("user.home"))) System.getProperty("user.home")))
downloadRetryInterval = Integer.parseInt(props.getProperty("downloadRetryInterval","5")) downloadRetryInterval = Integer.parseInt(props.getProperty("downloadRetryInterval","1"))
updateCheckInterval = Integer.parseInt(props.getProperty("updateCheckInterval","36")) updateCheckInterval = Integer.parseInt(props.getProperty("updateCheckInterval","24"))
shareDownloadedFiles = Boolean.parseBoolean(props.getProperty("shareDownloadedFiles","true")) shareDownloadedFiles = Boolean.parseBoolean(props.getProperty("shareDownloadedFiles","true"))
watchedDirectories = new HashSet<>() watchedDirectories = new HashSet<>()

View File

@@ -0,0 +1,4 @@
package com.muwire.core
class RouterDisconnectedEvent extends Event {
}

View File

@@ -3,6 +3,10 @@ package com.muwire.core.download
import com.muwire.core.connection.I2PConnector import com.muwire.core.connection.I2PConnector
import com.muwire.core.files.FileDownloadedEvent import com.muwire.core.files.FileDownloadedEvent
import com.muwire.core.files.FileHasher import com.muwire.core.files.FileHasher
import com.muwire.core.mesh.Mesh
import com.muwire.core.mesh.MeshManager
import com.muwire.core.trust.TrustLevel
import com.muwire.core.trust.TrustService
import com.muwire.core.util.DataUtil import com.muwire.core.util.DataUtil
import groovy.json.JsonBuilder import groovy.json.JsonBuilder
@@ -14,24 +18,33 @@ import net.i2p.util.ConcurrentHashSet
import com.muwire.core.EventBus import com.muwire.core.EventBus
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.MuWireSettings
import com.muwire.core.Persona import com.muwire.core.Persona
import com.muwire.core.UILoadedEvent import com.muwire.core.UILoadedEvent
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.Executor import java.util.concurrent.Executor
import java.util.concurrent.Executors import java.util.concurrent.Executors
public class DownloadManager { public class DownloadManager {
private final EventBus eventBus private final EventBus eventBus
private final TrustService trustService
private final MeshManager meshManager
private final MuWireSettings muSettings
private final I2PConnector connector private final I2PConnector connector
private final Executor executor private final Executor executor
private final File incompletes, home private final File incompletes, home
private final Persona me private final Persona me
private final Set<Downloader> downloaders = new ConcurrentHashSet<>() private final Map<InfoHash, Downloader> downloaders = new ConcurrentHashMap<>()
public DownloadManager(EventBus eventBus, I2PConnector connector, File home, Persona me) { public DownloadManager(EventBus eventBus, TrustService trustService, MeshManager meshManager, MuWireSettings muSettings,
I2PConnector connector, File home, Persona me) {
this.eventBus = eventBus this.eventBus = eventBus
this.trustService = trustService
this.meshManager = meshManager
this.muSettings = muSettings
this.connector = connector this.connector = connector
this.incompletes = new File(home,"incompletes") this.incompletes = new File(home,"incompletes")
this.home = home this.home = home
@@ -61,17 +74,27 @@ public class DownloadManager {
destinations.addAll(e.sources) destinations.addAll(e.sources)
destinations.remove(me.destination) destinations.remove(me.destination)
Pieces pieces = getPieces(infohash, size, pieceSize)
def downloader = new Downloader(eventBus, this, me, e.target, size, def downloader = new Downloader(eventBus, this, me, e.target, size,
infohash, pieceSize, connector, destinations, infohash, pieceSize, connector, destinations,
incompletes) incompletes, pieces)
downloaders.add(downloader) downloaders.put(infohash, downloader)
persistDownloaders() persistDownloaders()
executor.execute({downloader.download()} as Runnable) executor.execute({downloader.download()} as Runnable)
eventBus.publish(new DownloadStartedEvent(downloader : downloader)) eventBus.publish(new DownloadStartedEvent(downloader : downloader))
} }
public void onUIDownloadCancelledEvent(UIDownloadCancelledEvent e) { public void onUIDownloadCancelledEvent(UIDownloadCancelledEvent e) {
downloaders.remove(e.downloader) downloaders.remove(e.downloader.infoHash)
persistDownloaders()
}
public void onUIDownloadPausedEvent(UIDownloadPausedEvent e) {
persistDownloaders()
}
public void onUIDownloadResumedEvent(UIDownloadResumedEvent e) {
persistDownloaders() persistDownloaders()
} }
@@ -99,23 +122,54 @@ public class DownloadManager {
byte [] root = Base64.decode(json.hashRoot) byte [] root = Base64.decode(json.hashRoot)
infoHash = new InfoHash(root) infoHash = new InfoHash(root)
} }
Pieces pieces = getPieces(infoHash, (long)json.length, json.pieceSizePow2)
def downloader = new Downloader(eventBus, this, me, file, (long)json.length, def downloader = new Downloader(eventBus, this, me, file, (long)json.length,
infoHash, json.pieceSizePow2, connector, destinations, incompletes) infoHash, json.pieceSizePow2, connector, destinations, incompletes, pieces)
downloaders.add(downloader) if (json.paused != null)
downloader.download() downloader.paused = json.paused
downloaders.put(infoHash, downloader)
downloader.readPieces()
if (!downloader.paused)
downloader.download()
eventBus.publish(new DownloadStartedEvent(downloader : downloader)) eventBus.publish(new DownloadStartedEvent(downloader : downloader))
} }
} }
private Pieces getPieces(InfoHash infoHash, long length, int pieceSizePow2) {
int pieceSize = 0x1 << pieceSizePow2
int nPieces = (int)(length / pieceSize)
if (length % pieceSize != 0)
nPieces++
Mesh mesh = meshManager.getOrCreate(infoHash, nPieces)
mesh.pieces
}
void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
Downloader downloader = downloaders.get(e.infoHash)
if (downloader == null)
return
boolean ok = false
switch(trustService.getLevel(e.source.destination)) {
case TrustLevel.TRUSTED: ok = true; break
case TrustLevel.NEUTRAL: ok = muSettings.allowUntrusted; break
case TrustLevel.DISTRUSTED: ok = false; break
}
if (ok)
downloader.addSource(e.source.destination)
}
void onFileDownloadedEvent(FileDownloadedEvent e) { void onFileDownloadedEvent(FileDownloadedEvent e) {
downloaders.remove(e.downloader) downloaders.remove(e.downloader.infoHash)
persistDownloaders() persistDownloaders()
} }
private void persistDownloaders() { private void persistDownloaders() {
File downloadsFile = new File(home,"downloads.json") File downloadsFile = new File(home,"downloads.json")
downloadsFile.withPrintWriter { writer -> downloadsFile.withPrintWriter { writer ->
downloaders.each { downloader -> downloaders.values().each { downloader ->
if (!downloader.cancelled) { if (!downloader.cancelled) {
def json = [:] def json = [:]
json.file = Base64.encode(DataUtil.encodei18nString(downloader.file.getAbsolutePath())) json.file = Base64.encode(DataUtil.encodei18nString(downloader.file.getAbsolutePath()))
@@ -132,6 +186,8 @@ public class DownloadManager {
json.hashList = Base64.encode(infoHash.hashList) json.hashList = Base64.encode(infoHash.hashList)
else else
json.hashRoot = Base64.encode(infoHash.getRoot()) json.hashRoot = Base64.encode(infoHash.getRoot())
json.paused = downloader.paused
writer.println(JsonOutput.toJson(json)) writer.println(JsonOutput.toJson(json))
} }
} }
@@ -139,7 +195,7 @@ public class DownloadManager {
} }
public void shutdown() { public void shutdown() {
downloaders.each { it.stop() } downloaders.values().each { it.stop() }
Downloader.executorService.shutdownNow() Downloader.executorService.shutdownNow()
} }
} }

View File

@@ -3,8 +3,12 @@ package com.muwire.core.download;
import net.i2p.data.Base64 import net.i2p.data.Base64
import com.muwire.core.Constants import com.muwire.core.Constants
import com.muwire.core.EventBus
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.Persona
import com.muwire.core.connection.Endpoint import com.muwire.core.connection.Endpoint
import com.muwire.core.util.DataUtil
import static com.muwire.core.util.DataUtil.readTillRN import static com.muwire.core.util.DataUtil.readTillRN
import groovy.util.logging.Log import groovy.util.logging.Log
@@ -23,6 +27,7 @@ class DownloadSession {
private static int SAMPLES = 10 private static int SAMPLES = 10
private final EventBus eventBus
private final String meB64 private final String meB64
private final Pieces pieces private final Pieces pieces
private final InfoHash infoHash private final InfoHash infoHash
@@ -30,6 +35,7 @@ class DownloadSession {
private final File file private final File file
private final int pieceSize private final int pieceSize
private final long fileLength private final long fileLength
private final Set<Integer> available
private final MessageDigest digest private final MessageDigest digest
private final LinkedList<Long> timestamps = new LinkedList<>() private final LinkedList<Long> timestamps = new LinkedList<>()
@@ -37,8 +43,9 @@ class DownloadSession {
private ByteBuffer mapped private ByteBuffer mapped
DownloadSession(String meB64, Pieces pieces, InfoHash infoHash, Endpoint endpoint, File file, DownloadSession(EventBus eventBus, String meB64, Pieces pieces, InfoHash infoHash, Endpoint endpoint, File file,
int pieceSize, long fileLength) { int pieceSize, long fileLength, Set<Integer> available) {
this.eventBus = eventBus
this.meB64 = meB64 this.meB64 = meB64
this.pieces = pieces this.pieces = pieces
this.endpoint = endpoint this.endpoint = endpoint
@@ -46,6 +53,7 @@ class DownloadSession {
this.file = file this.file = file
this.pieceSize = pieceSize this.pieceSize = pieceSize
this.fileLength = fileLength this.fileLength = fileLength
this.available = available
try { try {
digest = MessageDigest.getInstance("SHA-256") digest = MessageDigest.getInstance("SHA-256")
} catch (NoSuchAlgorithmException impossible) { } catch (NoSuchAlgorithmException impossible) {
@@ -63,7 +71,11 @@ class DownloadSession {
OutputStream os = endpoint.getOutputStream() OutputStream os = endpoint.getOutputStream()
InputStream is = endpoint.getInputStream() InputStream is = endpoint.getInputStream()
int piece = pieces.claim() int piece
if (available.isEmpty())
piece = pieces.claim()
else
piece = pieces.claim(available)
if (piece == -1) if (piece == -1)
return false return false
boolean unclaim = true boolean unclaim = true
@@ -79,45 +91,79 @@ class DownloadSession {
try { try {
os.write("GET $root\r\n".getBytes(StandardCharsets.US_ASCII)) os.write("GET $root\r\n".getBytes(StandardCharsets.US_ASCII))
os.write("Range: $start-$end\r\n".getBytes(StandardCharsets.US_ASCII)) os.write("Range: $start-$end\r\n".getBytes(StandardCharsets.US_ASCII))
os.write("X-Persona: $meB64\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) os.write("X-Persona: $meB64\r\n".getBytes(StandardCharsets.US_ASCII))
String xHave = DataUtil.encodeXHave(pieces.getDownloaded(), pieces.nPieces)
os.write("X-Have: $xHave\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
os.flush() os.flush()
String code = readTillRN(is) String codeString = readTillRN(is)
if (code.startsWith("404 ")) { int space = codeString.indexOf(' ')
if (space > 0)
codeString = codeString.substring(0, space)
int code = Integer.parseInt(codeString.trim())
if (code == 404) {
log.warning("file not found") log.warning("file not found")
endpoint.close() endpoint.close()
return false return false
} }
if (code.startsWith("416 ")) { if (!(code == 200 || code == 416)) {
log.warning("range $start-$end cannot be satisfied")
return // leave endpoint open
}
if (!code.startsWith("200 ")) {
log.warning("unknown code $code") log.warning("unknown code $code")
endpoint.close() endpoint.close()
return false return false
} }
// parse all headers // parse all headers
Set<String> headers = new HashSet<>() Map<String,String> headers = new HashMap<>()
String header String header
while((header = readTillRN(is)) != "" && headers.size() < Constants.MAX_HEADERS) while((header = readTillRN(is)) != "" && headers.size() < Constants.MAX_HEADERS) {
headers.add(header) int colon = header.indexOf(':')
if (colon == -1 || colon == header.length() - 1)
long receivedStart = -1 throw new IOException("invalid header $header")
long receivedEnd = -1 String key = header.substring(0, colon)
for (String receivedHeader : headers) { String value = header.substring(colon + 1)
def group = (receivedHeader =~ /^Content-Range: (\d+)-(\d+)$/) headers[key] = value.trim()
if (group.size() != 1) {
log.info("ignoring header $receivedHeader")
continue
}
receivedStart = Long.parseLong(group[0][1])
receivedEnd = Long.parseLong(group[0][2])
} }
// prase X-Alt if present
if (headers.containsKey("X-Alt")) {
headers["X-Alt"].split(",").each {
if (it.length() > 0) {
byte [] raw = Base64.decode(it)
Persona source = new Persona(new ByteArrayInputStream(raw))
eventBus.publish(new SourceDiscoveredEvent(infoHash : infoHash, source : source))
}
}
}
// parse X-Have if present
if (headers.containsKey("X-Have")) {
DataUtil.decodeXHave(headers["X-Have"]).each {
available.add(it)
}
if (!available.contains(piece))
return true // try again next time
} else {
if (code != 200)
throw new IOException("Code $code but no X-Have")
available.clear()
}
if (code != 200)
return true
String range = headers["Content-Range"]
if (range == null)
throw new IOException("Code 200 but no Content-Range")
def group = (range =~ /^(\d+)-(\d+)$/)
if (group.size() != 1)
throw new IOException("invalid Content-Range header $range")
long receivedStart = Long.parseLong(group[0][1])
long receivedEnd = Long.parseLong(group[0][2])
if (receivedStart != start || receivedEnd != end) { if (receivedStart != start || receivedEnd != end) {
log.warning("We don't support mismatching ranges yet") log.warning("We don't support mismatching ranges yet")
endpoint.close() endpoint.close()

View File

@@ -25,7 +25,7 @@ import net.i2p.util.ConcurrentHashSet
@Log @Log
public class Downloader { public class Downloader {
public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, CANCELLED, FINISHED } public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, CANCELLED, PAUSED, FINISHED }
private enum WorkerState { CONNECTING, HASHLIST, DOWNLOADING, FINISHED} private enum WorkerState { CONNECTING, HASHLIST, DOWNLOADING, FINISHED}
private static final ExecutorService executorService = Executors.newCachedThreadPool({r -> private static final ExecutorService executorService = Executors.newCachedThreadPool({r ->
@@ -53,14 +53,14 @@ public class Downloader {
private final Set<Destination> successfulDestinations = new ConcurrentHashSet<>() private final Set<Destination> successfulDestinations = new ConcurrentHashSet<>()
private volatile boolean cancelled private volatile boolean cancelled, paused
private final AtomicBoolean eventFired = new AtomicBoolean() private final AtomicBoolean eventFired = new AtomicBoolean()
private boolean piecesFileClosed private boolean piecesFileClosed
public Downloader(EventBus eventBus, DownloadManager downloadManager, public Downloader(EventBus eventBus, DownloadManager downloadManager,
Persona me, File file, long length, InfoHash infoHash, Persona me, File file, long length, InfoHash infoHash,
int pieceSizePow2, I2PConnector connector, Set<Destination> destinations, int pieceSizePow2, I2PConnector connector, Set<Destination> destinations,
File incompletes) { File incompletes, Pieces pieces) {
this.eventBus = eventBus this.eventBus = eventBus
this.me = me this.me = me
this.downloadManager = downloadManager this.downloadManager = downloadManager
@@ -73,15 +73,8 @@ public class Downloader {
this.incompleteFile = new File(incompletes, file.getName()+".part") this.incompleteFile = new File(incompletes, file.getName()+".part")
this.pieceSizePow2 = pieceSizePow2 this.pieceSizePow2 = pieceSizePow2
this.pieceSize = 1 << pieceSizePow2 this.pieceSize = 1 << pieceSizePow2
this.pieces = pieces
int nPieces this.nPieces = pieces.nPieces
if (length % pieceSize == 0)
nPieces = length / pieceSize
else
nPieces = length / pieceSize + 1
this.nPieces = nPieces
pieces = new Pieces(nPieces, Constants.DOWNLOAD_SEQUENTIAL_RATIO)
} }
public synchronized InfoHash getInfoHash() { public synchronized InfoHash getInfoHash() {
@@ -143,6 +136,9 @@ public class Downloader {
public DownloadState getCurrentState() { public DownloadState getCurrentState() {
if (cancelled) if (cancelled)
return DownloadState.CANCELLED return DownloadState.CANCELLED
if (paused)
return DownloadState.PAUSED
boolean allFinished = true boolean allFinished = true
activeWorkers.values().each { activeWorkers.values().each {
allFinished &= it.currentState == WorkerState.FINISHED allFinished &= it.currentState == WorkerState.FINISHED
@@ -187,6 +183,12 @@ public class Downloader {
piecesFile.delete() piecesFile.delete()
} }
incompleteFile.delete() incompleteFile.delete()
pieces.clearAll()
}
public void pause() {
paused = true
stop()
} }
void stop() { void stop() {
@@ -205,6 +207,8 @@ public class Downloader {
} }
public void resume() { public void resume() {
paused = false
readPieces()
destinations.each { destination -> destinations.each { destination ->
def worker = activeWorkers.get(destination) def worker = activeWorkers.get(destination)
if (worker != null) { if (worker != null) {
@@ -221,12 +225,21 @@ public class Downloader {
} }
} }
void addSource(Destination d) {
if (activeWorkers.containsKey(d))
return
DownloadWorker newWorker = new DownloadWorker(d)
activeWorkers.put(d, newWorker)
executorService.submit(newWorker)
}
class DownloadWorker implements Runnable { class DownloadWorker implements Runnable {
private final Destination destination private final Destination destination
private volatile WorkerState currentState private volatile WorkerState currentState
private volatile Thread downloadThread private volatile Thread downloadThread
private Endpoint endpoint private Endpoint endpoint
private volatile DownloadSession currentSession private volatile DownloadSession currentSession
private final Set<Integer> available = new HashSet<>()
DownloadWorker(Destination destination) { DownloadWorker(Destination destination) {
this.destination = destination this.destination = destination
@@ -247,7 +260,8 @@ public class Downloader {
currentState = WorkerState.DOWNLOADING currentState = WorkerState.DOWNLOADING
boolean requestPerformed boolean requestPerformed
while(!pieces.isComplete()) { while(!pieces.isComplete()) {
currentSession = new DownloadSession(me.toBase64(), pieces, getInfoHash(), endpoint, incompleteFile, pieceSize, length) currentSession = new DownloadSession(eventBus, me.toBase64(), pieces, getInfoHash(),
endpoint, incompleteFile, pieceSize, length, available)
requestPerformed = currentSession.request() requestPerformed = currentSession.request()
if (!requestPerformed) if (!requestPerformed)
break break

View File

@@ -38,6 +38,18 @@ class Pieces {
} }
} }
synchronized int claim(Set<Integer> available) {
for (int i = claimed.nextSetBit(0); i >= 0; i = claimed.nextSetBit(i+1))
available.remove(i)
if (available.isEmpty())
return -1
List<Integer> toList = available.toList()
Collections.shuffle(toList)
int rv = toList[0]
claimed.set(rv)
rv
}
synchronized def getDownloaded() { synchronized def getDownloaded() {
def rv = [] def rv = []
for (int i = done.nextSetBit(0); i >= 0; i = done.nextSetBit(i+1)) { for (int i = done.nextSetBit(0); i >= 0; i = done.nextSetBit(i+1)) {
@@ -62,4 +74,13 @@ class Pieces {
synchronized int donePieces() { synchronized int donePieces() {
done.cardinality() done.cardinality()
} }
synchronized boolean isDownloaded(int piece) {
done.get(piece)
}
synchronized void clearAll() {
done.clear()
claimed.clear()
}
} }

View File

@@ -0,0 +1,10 @@
package com.muwire.core.download
import com.muwire.core.Event
import com.muwire.core.InfoHash
import com.muwire.core.Persona
class SourceDiscoveredEvent extends Event {
InfoHash infoHash
Persona source
}

View File

@@ -0,0 +1,6 @@
package com.muwire.core.download
import com.muwire.core.Event
class UIDownloadPausedEvent extends Event {
}

View File

@@ -0,0 +1,6 @@
package com.muwire.core.download
import com.muwire.core.Event
class UIDownloadResumedEvent extends Event {
}

View File

@@ -0,0 +1,28 @@
package com.muwire.core.mesh
import com.muwire.core.InfoHash
import com.muwire.core.Persona
import com.muwire.core.download.Pieces
import net.i2p.data.Destination
import net.i2p.util.ConcurrentHashSet
class Mesh {
private final InfoHash infoHash
private final Set<Persona> sources = new ConcurrentHashSet<>()
private final Pieces pieces
Mesh(InfoHash infoHash, Pieces pieces) {
this.infoHash = infoHash
this.pieces = pieces
}
Set<Persona> getRandom(int n, Persona exclude) {
List<Persona> tmp = new ArrayList<>(sources)
tmp.remove(exclude)
Collections.shuffle(tmp)
if (tmp.size() < n)
return tmp
tmp[0..n-1]
}
}

View File

@@ -0,0 +1,43 @@
package com.muwire.core.mesh
import com.muwire.core.Constants
import com.muwire.core.InfoHash
import com.muwire.core.download.Pieces
import com.muwire.core.download.SourceDiscoveredEvent
import com.muwire.core.files.FileManager
class MeshManager {
private final Map<InfoHash, Mesh> meshes = Collections.synchronizedMap(new HashMap<>())
private final FileManager fileManager
MeshManager(FileManager fileManager) {
this.fileManager = fileManager
}
Mesh get(InfoHash infoHash) {
meshes.get(infoHash)
}
Mesh getOrCreate(InfoHash infoHash, int nPieces) {
synchronized(meshes) {
if (meshes.containsKey(infoHash))
return meshes.get(infoHash)
Pieces pieces = new Pieces(nPieces, Constants.DOWNLOAD_SEQUENTIAL_RATIO)
if (fileManager.rootToFiles.containsKey(infoHash)) {
for (int i = 0; i < nPieces; i++)
pieces.markDownloaded(i)
}
Mesh rv = new Mesh(infoHash, pieces)
meshes.put(infoHash, rv)
return rv
}
}
void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
Mesh mesh = meshes.get(e.infoHash)
if (mesh == null)
return
mesh.sources.add(e.source)
}
}

View File

@@ -11,6 +11,7 @@ import java.util.concurrent.Executor
import java.util.concurrent.Executors import java.util.concurrent.Executors
import java.util.concurrent.ThreadFactory import java.util.concurrent.ThreadFactory
import java.util.concurrent.atomic.AtomicInteger import java.util.concurrent.atomic.AtomicInteger
import java.util.logging.Level
import java.util.stream.Collectors import java.util.stream.Collectors
import com.muwire.core.DownloadedFile import com.muwire.core.DownloadedFile
@@ -83,50 +84,54 @@ class ResultsSender {
@Override @Override
public void run() { public void run() {
byte [] tmp = new byte[InfoHash.SIZE]
JsonOutput jsonOutput = new JsonOutput()
Endpoint endpoint = null;
try { try {
endpoint = connector.connect(target) byte [] tmp = new byte[InfoHash.SIZE]
DataOutputStream os = new DataOutputStream(endpoint.getOutputStream()) JsonOutput jsonOutput = new JsonOutput()
os.write("POST $uuid\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) Endpoint endpoint = null;
me.write(os) try {
os.writeShort((short)results.length) endpoint = connector.connect(target)
results.each { DataOutputStream os = new DataOutputStream(endpoint.getOutputStream())
byte [] name = it.getFile().getName().getBytes(StandardCharsets.UTF_8) os.write("POST $uuid\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
def baos = new ByteArrayOutputStream() me.write(os)
def daos = new DataOutputStream(baos) os.writeShort((short)results.length)
daos.writeShort((short) name.length) results.each {
daos.write(name) byte [] name = it.getFile().getName().getBytes(StandardCharsets.UTF_8)
daos.flush() def baos = new ByteArrayOutputStream()
String encodedName = Base64.encode(baos.toByteArray()) def daos = new DataOutputStream(baos)
def obj = [:] daos.writeShort((short) name.length)
obj.type = "Result" daos.write(name)
obj.version = oobInfohash ? 2 : 1 daos.flush()
obj.name = encodedName String encodedName = Base64.encode(baos.toByteArray())
obj.infohash = Base64.encode(it.getInfoHash().getRoot()) def obj = [:]
obj.size = it.getFile().length() obj.type = "Result"
obj.pieceSize = it.getPieceSize() obj.version = oobInfohash ? 2 : 1
if (!oobInfohash) { obj.name = encodedName
byte [] hashList = it.getInfoHash().getHashList() obj.infohash = Base64.encode(it.getInfoHash().getRoot())
def hashListB64 = [] obj.size = it.getFile().length()
for (int i = 0; i < hashList.length / InfoHash.SIZE; i++) { obj.pieceSize = it.getPieceSize()
System.arraycopy(hashList, InfoHash.SIZE * i, tmp, 0, InfoHash.SIZE) if (!oobInfohash) {
hashListB64 << Base64.encode(tmp) byte [] hashList = it.getInfoHash().getHashList()
def hashListB64 = []
for (int i = 0; i < hashList.length / InfoHash.SIZE; i++) {
System.arraycopy(hashList, InfoHash.SIZE * i, tmp, 0, InfoHash.SIZE)
hashListB64 << Base64.encode(tmp)
}
obj.hashList = hashListB64
} }
obj.hashList = hashListB64
if (it instanceof DownloadedFile)
obj.sources = it.sources.stream().map({dest -> dest.toBase64()}).collect(Collectors.toSet())
def json = jsonOutput.toJson(obj)
os.writeShort((short)json.length())
os.write(json.getBytes(StandardCharsets.US_ASCII))
} }
os.flush()
if (it instanceof DownloadedFile) } finally {
obj.sources = it.sources.stream().map({dest -> dest.toBase64()}).collect(Collectors.toSet()) endpoint?.close()
def json = jsonOutput.toJson(obj)
os.writeShort((short)json.length())
os.write(json.getBytes(StandardCharsets.US_ASCII))
} }
os.flush() } catch (Exception e) {
} finally { log.log(Level.WARNING, "problem sending results",e)
endpoint?.close()
} }
} }
} }

View File

@@ -33,7 +33,10 @@ class SearchIndex {
private static String[] split(String source) { private static String[] split(String source) {
source = source.replaceAll(Constants.SPLIT_PATTERN, " ").toLowerCase() source = source.replaceAll(Constants.SPLIT_PATTERN, " ").toLowerCase()
source.split(" ") String [] split = source.split(" ")
def rv = []
split.each { if (it.length() > 0) rv << it }
rv.toArray(new String[0])
} }
String[] search(List<String> terms) { String[] search(List<String> terms) {

View File

@@ -2,4 +2,5 @@ package com.muwire.core.upload
class ContentRequest extends Request { class ContentRequest extends Request {
Range range Range range
boolean have
} }

View File

@@ -5,33 +5,57 @@ import java.nio.channels.FileChannel
import java.nio.charset.StandardCharsets import java.nio.charset.StandardCharsets
import java.nio.file.Files import java.nio.file.Files
import java.nio.file.StandardOpenOption import java.nio.file.StandardOpenOption
import java.util.stream.Collectors
import com.muwire.core.Persona
import com.muwire.core.connection.Endpoint import com.muwire.core.connection.Endpoint
import com.muwire.core.mesh.Mesh
import com.muwire.core.util.DataUtil
import net.i2p.data.Destination
class ContentUploader extends Uploader { class ContentUploader extends Uploader {
private final File file private final File file
private final ContentRequest request private final ContentRequest request
private final Mesh mesh
private final int pieceSize
ContentUploader(File file, ContentRequest request, Endpoint endpoint) { ContentUploader(File file, ContentRequest request, Endpoint endpoint, Mesh mesh, int pieceSize) {
super(endpoint) super(endpoint)
this.file = file this.file = file
this.request = request this.request = request
this.mesh = mesh
this.pieceSize = pieceSize
} }
@Override @Override
void respond() { void respond() {
OutputStream os = endpoint.getOutputStream() OutputStream os = endpoint.getOutputStream()
Range range = request.getRange() Range range = request.getRange()
if (range.start >= file.length() || range.end >= file.length()) { boolean satisfiable = true
os.write("416 Range Not Satisfiable\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) final long length = file.length()
if (range.start >= length || range.end >= length)
satisfiable = false
if (satisfiable) {
int startPiece = range.start / (0x1 << pieceSize)
int endPiece = range.end / (0x1 << pieceSize)
for (int i = startPiece; i <= endPiece; i++)
satisfiable &= mesh.pieces.isDownloaded(i)
}
if (!satisfiable) {
os.write("416 Range Not Satisfiable\r\n".getBytes(StandardCharsets.US_ASCII))
writeMesh(request.downloader)
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
os.flush() os.flush()
return return
} }
os.write("200 OK\r\n".getBytes(StandardCharsets.US_ASCII)) os.write("200 OK\r\n".getBytes(StandardCharsets.US_ASCII))
os.write("Content-Range: $range.start-$range.end\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) os.write("Content-Range: $range.start-$range.end\r\n".getBytes(StandardCharsets.US_ASCII))
writeMesh(request.downloader)
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
FileChannel channel FileChannel channel
try { try {
channel = Files.newByteChannel(file.toPath(), EnumSet.of(StandardOpenOption.READ)) channel = Files.newByteChannel(file.toPath(), EnumSet.of(StandardOpenOption.READ))
@@ -50,6 +74,17 @@ class ContentUploader extends Uploader {
endpoint.getOutputStream().flush() endpoint.getOutputStream().flush()
} }
} }
private void writeMesh(Persona toExclude) {
String xHave = DataUtil.encodeXHave(mesh.pieces.getDownloaded(), mesh.pieces.nPieces)
endpoint.getOutputStream().write("X-Have: $xHave\r\n".getBytes(StandardCharsets.US_ASCII))
Set<Persona> sources = mesh.getRandom(3, toExclude)
if (!sources.isEmpty()) {
String xAlts = sources.stream().map({ it.toBase64() }).collect(Collectors.joining(","))
endpoint.getOutputStream().write("X-Alt: $xAlts\r\n".getBytes(StandardCharsets.US_ASCII))
}
}
@Override @Override
public String getName() { public String getName() {

View File

@@ -5,6 +5,7 @@ import java.nio.charset.StandardCharsets
import com.muwire.core.Constants import com.muwire.core.Constants
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.Persona import com.muwire.core.Persona
import com.muwire.core.util.DataUtil
import groovy.util.logging.Log import groovy.util.logging.Log
import net.i2p.data.Base64 import net.i2p.data.Base64
@@ -48,8 +49,14 @@ class Request {
def decoded = Base64.decode(encoded) def decoded = Base64.decode(encoded)
downloader = new Persona(new ByteArrayInputStream(decoded)) downloader = new Persona(new ByteArrayInputStream(decoded))
} }
boolean have = false
if (headers.containsKey("X-Have")) {
def encoded = headers["X-Have"].trim()
have = DataUtil.decodeXHave(encoded).size() > 0
}
new ContentRequest( infoHash : infoHash, range : new Range(start, end), new ContentRequest( infoHash : infoHash, range : new Range(start, end),
headers : headers, downloader : downloader) headers : headers, downloader : downloader, have : have)
} }
static Request parseHashListRequest(InfoHash infoHash, InputStream is) throws IOException { static Request parseHashListRequest(InfoHash infoHash, InputStream is) throws IOException {

View File

@@ -6,7 +6,12 @@ import com.muwire.core.EventBus
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.SharedFile import com.muwire.core.SharedFile
import com.muwire.core.connection.Endpoint import com.muwire.core.connection.Endpoint
import com.muwire.core.download.DownloadManager
import com.muwire.core.download.Downloader
import com.muwire.core.download.SourceDiscoveredEvent
import com.muwire.core.files.FileManager import com.muwire.core.files.FileManager
import com.muwire.core.mesh.Mesh
import com.muwire.core.mesh.MeshManager
import groovy.util.logging.Log import groovy.util.logging.Log
import net.i2p.data.Base64 import net.i2p.data.Base64
@@ -15,12 +20,17 @@ import net.i2p.data.Base64
public class UploadManager { public class UploadManager {
private final EventBus eventBus private final EventBus eventBus
private final FileManager fileManager private final FileManager fileManager
private final MeshManager meshManager
private final DownloadManager downloadManager
public UploadManager() {} public UploadManager() {}
public UploadManager(EventBus eventBus, FileManager fileManager) { public UploadManager(EventBus eventBus, FileManager fileManager,
MeshManager meshManager, DownloadManager downloadManager) {
this.eventBus = eventBus this.eventBus = eventBus
this.fileManager = fileManager this.fileManager = fileManager
this.meshManager = meshManager
this.downloadManager = downloadManager
} }
public void processGET(Endpoint e) throws IOException { public void processGET(Endpoint e) throws IOException {
@@ -44,8 +54,10 @@ public class UploadManager {
log.info("Responding to upload request for root $infoHashString") log.info("Responding to upload request for root $infoHashString")
byte [] infoHashRoot = Base64.decode(infoHashString) byte [] infoHashRoot = Base64.decode(infoHashString)
InfoHash infoHash = new InfoHash(infoHashRoot)
Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot) Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot)
if (sharedFiles == null || sharedFiles.isEmpty()) { Downloader downloader = downloadManager.downloaders.get(infoHash)
if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
log.info "file not found" log.info "file not found"
e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
e.getOutputStream().flush() e.getOutputStream().flush()
@@ -61,13 +73,31 @@ public class UploadManager {
return return
} }
Request request = Request.parseContentRequest(new InfoHash(infoHashRoot), e.getInputStream()) ContentRequest request = Request.parseContentRequest(infoHash, e.getInputStream())
if (request.downloader != null && request.downloader.destination != e.destination) { if (request.downloader != null && request.downloader.destination != e.destination) {
log.info("Downloader persona doesn't match their destination") log.info("Downloader persona doesn't match their destination")
e.close() e.close()
return return
} }
Uploader uploader = new ContentUploader(sharedFiles.iterator().next().file, request, e)
if (request.have)
eventBus.publish(new SourceDiscoveredEvent(infoHash : request.infoHash, source : request.downloader))
Mesh mesh
File file
int pieceSize
if (downloader != null) {
mesh = meshManager.get(infoHash)
file = downloader.incompleteFile
pieceSize = downloader.pieceSizePow2
} else {
SharedFile sharedFile = sharedFiles.iterator().next();
mesh = meshManager.getOrCreate(request.infoHash, sharedFile.NPieces)
file = sharedFile.file
pieceSize = sharedFile.pieceSize
}
Uploader uploader = new ContentUploader(file, request, e, mesh, pieceSize)
eventBus.publish(new UploadEvent(uploader : uploader)) eventBus.publish(new UploadEvent(uploader : uploader))
try { try {
uploader.respond() uploader.respond()
@@ -85,8 +115,10 @@ public class UploadManager {
log.info("Responding to hashlist request for root $infoHashString") log.info("Responding to hashlist request for root $infoHashString")
byte [] infoHashRoot = Base64.decode(infoHashString) byte [] infoHashRoot = Base64.decode(infoHashString)
InfoHash infoHash = new InfoHash(infoHashRoot)
Downloader downloader = downloadManager.downloaders.get(infoHash)
Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot) Set<SharedFile> sharedFiles = fileManager.getSharedFiles(infoHashRoot)
if (sharedFiles == null || sharedFiles.isEmpty()) { if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
log.info "file not found" log.info "file not found"
e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
e.getOutputStream().flush() e.getOutputStream().flush()
@@ -102,13 +134,30 @@ public class UploadManager {
return return
} }
Request request = Request.parseHashListRequest(new InfoHash(infoHashRoot), e.getInputStream()) Request request = Request.parseHashListRequest(infoHash, e.getInputStream())
if (request.downloader != null && request.downloader.destination != e.destination) { if (request.downloader != null && request.downloader.destination != e.destination) {
log.info("Downloader persona doesn't match their destination") log.info("Downloader persona doesn't match their destination")
e.close() e.close()
return return
} }
Uploader uploader = new HashListUploader(e, sharedFiles.iterator().next().infoHash, request)
InfoHash fullInfoHash
if (downloader == null) {
fullInfoHash = sharedFiles.iterator().next().infoHash
} else {
byte [] hashList = downloader.getInfoHash().getHashList()
if (hashList != null && hashList.length > 0)
fullInfoHash = downloader.getInfoHash()
else {
log.info("infohash not found in downloader")
e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
e.getOutputStream().flush()
e.close()
return
}
}
Uploader uploader = new HashListUploader(e, fullInfoHash, request)
eventBus.publish(new UploadEvent(uploader : uploader)) eventBus.publish(new UploadEvent(uploader : uploader))
try { try {
uploader.respond() uploader.respond()
@@ -130,8 +179,10 @@ public class UploadManager {
log.info("Responding to upload request for root $infoHashString") log.info("Responding to upload request for root $infoHashString")
infoHashRoot = Base64.decode(infoHashString) infoHashRoot = Base64.decode(infoHashString)
infoHash = new InfoHash(infoHashRoot)
sharedFiles = fileManager.getSharedFiles(infoHashRoot) sharedFiles = fileManager.getSharedFiles(infoHashRoot)
if (sharedFiles == null || sharedFiles.isEmpty()) { downloader = downloadManager.downloaders.get(infoHash)
if (downloader == null && (sharedFiles == null || sharedFiles.isEmpty())) {
log.info "file not found" log.info "file not found"
e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII)) e.getOutputStream().write("404 File Not Found\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
e.getOutputStream().flush() e.getOutputStream().flush()
@@ -153,7 +204,25 @@ public class UploadManager {
e.close() e.close()
return return
} }
uploader = new ContentUploader(sharedFiles.iterator().next().file, request, e)
if (request.have)
eventBus.publish(new SourceDiscoveredEvent(infoHash : request.infoHash, source : request.downloader))
Mesh mesh
File file
int pieceSize
if (downloader != null) {
mesh = meshManager.get(infoHash)
file = downloader.incompleteFile
pieceSize = downloader.pieceSizePow2
} else {
SharedFile sharedFile = sharedFiles.iterator().next();
mesh = meshManager.getOrCreate(request.infoHash, sharedFile.NPieces)
file = sharedFile.file
pieceSize = sharedFile.pieceSize
}
uploader = new ContentUploader(file, request, e, mesh, pieceSize)
eventBus.publish(new UploadEvent(uploader : uploader)) eventBus.publish(new UploadEvent(uploader : uploader))
try { try {
uploader.respond() uploader.respond()

View File

@@ -4,6 +4,8 @@ import java.nio.charset.StandardCharsets
import com.muwire.core.Constants import com.muwire.core.Constants
import net.i2p.data.Base64
class DataUtil { class DataUtil {
private final static int MAX_SHORT = (0x1 << 16) - 1 private final static int MAX_SHORT = (0x1 << 16) - 1
@@ -79,4 +81,32 @@ class DataUtil {
} }
new String(baos.toByteArray(), StandardCharsets.US_ASCII) new String(baos.toByteArray(), StandardCharsets.US_ASCII)
} }
public static String encodeXHave(List<Integer> pieces, int totalPieces) {
int bytes = totalPieces / 8
if (totalPieces % 8 != 0)
bytes++
byte[] raw = new byte[bytes]
pieces.each {
int byteIdx = it / 8
int offset = it % 8
int mask = 0x80 >>> offset
raw[byteIdx] |= mask
}
Base64.encode(raw)
}
public static List<Integer> decodeXHave(String xHave) {
byte [] availablePieces = Base64.decode(xHave)
List<Integer> available = new ArrayList<>()
availablePieces.eachWithIndex {b, i ->
for (int j = 0; j < 8 ; j++) {
byte mask = 0x80 >>> j
if ((b & mask) == mask) {
available.add(i * 8 + j)
}
}
}
available
}
} }

View File

@@ -26,6 +26,15 @@ public class SharedFile {
return pieceSize; return pieceSize;
} }
public int getNPieces() {
long length = file.length();
int rawPieceSize = 0x1 << pieceSize;
int rv = (int) (length / rawPieceSize);
if (length % pieceSize != 0)
rv++;
return rv;
}
@Override @Override
public int hashCode() { public int hashCode() {
return file.hashCode() ^ infoHash.hashCode(); return file.hashCode() ^ infoHash.hashCode();

View File

@@ -1,17 +1,26 @@
package com.muwire.core.download package com.muwire.core.download
import static org.junit.Assert.fail
import org.junit.After import org.junit.After
import org.junit.Before
import org.junit.Test import org.junit.Test
import com.muwire.core.EventBus
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.Persona
import com.muwire.core.Personas
import com.muwire.core.connection.Endpoint import com.muwire.core.connection.Endpoint
import com.muwire.core.files.FileHasher import com.muwire.core.files.FileHasher
import static com.muwire.core.util.DataUtil.readTillRN import static com.muwire.core.util.DataUtil.readTillRN
import static com.muwire.core.util.DataUtil.encodeXHave
import net.i2p.data.Base64 import net.i2p.data.Base64
import net.i2p.util.ConcurrentHashSet
class DownloadSessionTest { class DownloadSessionTest {
private EventBus eventBus
private File source, target private File source, target
private InfoHash infoHash private InfoHash infoHash
private Endpoint endpoint private Endpoint endpoint
@@ -24,6 +33,16 @@ class DownloadSessionTest {
private InputStream fromDownloader, fromUploader private InputStream fromDownloader, fromUploader
private OutputStream toDownloader, toUploader private OutputStream toDownloader, toUploader
private volatile boolean performed
private Set<Integer> available = new ConcurrentHashSet<>()
private volatile IOException thrown
@Before
public void setUp() {
eventBus = new EventBus()
}
private void initSession(int size, def claimedPieces = []) { private void initSession(int size, def claimedPieces = []) {
Random r = new Random() Random r = new Random()
byte [] content = new byte[size] byte [] content = new byte[size]
@@ -56,12 +75,20 @@ class DownloadSessionTest {
toUploader = new PipedOutputStream(fromDownloader) toUploader = new PipedOutputStream(fromDownloader)
endpoint = new Endpoint(null, fromUploader, toUploader, null) endpoint = new Endpoint(null, fromUploader, toUploader, null)
session = new DownloadSession("",pieces, infoHash, endpoint, target, pieceSize, size) session = new DownloadSession(eventBus, "",pieces, infoHash, endpoint, target, pieceSize, size, available)
downloadThread = new Thread( { session.request() } as Runnable) downloadThread = new Thread( { perform() } as Runnable)
downloadThread.setDaemon(true) downloadThread.setDaemon(true)
downloadThread.start() downloadThread.start()
} }
private void perform() {
try {
performed = session.request()
} catch (IOException e) {
thrown = e
}
}
@After @After
public void teardown() { public void teardown() {
source?.delete() source?.delete()
@@ -76,6 +103,7 @@ class DownloadSessionTest {
assert "GET $rootBase64" == readTillRN(fromDownloader) assert "GET $rootBase64" == readTillRN(fromDownloader)
assert "Range: 0-19" == readTillRN(fromDownloader) assert "Range: 0-19" == readTillRN(fromDownloader)
readTillRN(fromDownloader) readTillRN(fromDownloader)
readTillRN(fromDownloader)
assert "" == readTillRN(fromDownloader) assert "" == readTillRN(fromDownloader)
toDownloader.write("200 OK\r\n".bytes) toDownloader.write("200 OK\r\n".bytes)
@@ -87,6 +115,9 @@ class DownloadSessionTest {
assert pieces.isComplete() assert pieces.isComplete()
assert target.bytes == source.bytes assert target.bytes == source.bytes
assert performed
assert available.isEmpty()
assert thrown == null
} }
@Test @Test
@@ -98,6 +129,7 @@ class DownloadSessionTest {
assert "GET $rootBase64" == readTillRN(fromDownloader) assert "GET $rootBase64" == readTillRN(fromDownloader)
readTillRN(fromDownloader) readTillRN(fromDownloader)
readTillRN(fromDownloader) readTillRN(fromDownloader)
readTillRN(fromDownloader)
assert "" == readTillRN(fromDownloader) assert "" == readTillRN(fromDownloader)
toDownloader.write("200 OK\r\n".bytes) toDownloader.write("200 OK\r\n".bytes)
@@ -108,6 +140,9 @@ class DownloadSessionTest {
Thread.sleep(150) Thread.sleep(150)
assert pieces.isComplete() assert pieces.isComplete()
assert target.bytes == source.bytes assert target.bytes == source.bytes
assert performed
assert available.isEmpty()
assert thrown == null
} }
@Test @Test
@@ -125,6 +160,7 @@ class DownloadSessionTest {
assert (start == 0 && end == ((1 << pieceSize) - 1)) || assert (start == 0 && end == ((1 << pieceSize) - 1)) ||
(start == (1 << pieceSize) && end == (1 << pieceSize)) (start == (1 << pieceSize) && end == (1 << pieceSize))
readTillRN(fromDownloader)
readTillRN(fromDownloader) readTillRN(fromDownloader)
assert "" == readTillRN(fromDownloader) assert "" == readTillRN(fromDownloader)
@@ -138,6 +174,9 @@ class DownloadSessionTest {
Thread.sleep(150) Thread.sleep(150)
assert !pieces.isComplete() assert !pieces.isComplete()
assert 1 == pieces.donePieces() assert 1 == pieces.donePieces()
assert performed
assert available.isEmpty()
assert thrown == null
} }
@Test @Test
@@ -145,7 +184,10 @@ class DownloadSessionTest {
initSession(20, [0]) initSession(20, [0])
long now = System.currentTimeMillis() long now = System.currentTimeMillis()
downloadThread.join(100) downloadThread.join(100)
assert 100 > (System.currentTimeMillis() - now) assert 100 >= (System.currentTimeMillis() - now)
assert !performed
assert available.isEmpty()
assert thrown == null
} }
@Test @Test
@@ -164,4 +206,128 @@ class DownloadSessionTest {
assert pieces.claimed.get(0) assert pieces.claimed.get(0)
assert start == 0 && end == (1 << pieceSize) - 1 assert start == 0 && end == (1 << pieceSize) - 1
} }
@Test
public void test416NoHave() {
initSession(20)
readAllHeaders(fromDownloader)
toDownloader.write("416 don't have it\r\n\r\n".bytes)
toDownloader.flush()
Thread.sleep(150)
assert !performed
assert available.isEmpty()
assert thrown != null
}
@Test
public void test416Have() {
initSession(20)
readAllHeaders(fromDownloader)
toDownloader.write("416 don't have it\r\n".bytes)
toDownloader.write("X-Have: ${encodeXHave([0], 1)}\r\n\r\n".bytes)
toDownloader.flush()
Thread.sleep(150)
assert performed
assert available.contains(0)
assert thrown == null
}
@Test
public void test416Have2Pieces() {
int pieceSize = FileHasher.getPieceSize(1)
int size = (1 << pieceSize) + 1
initSession(size)
readAllHeaders(fromDownloader)
toDownloader.write("416 don't have it\r\n".bytes)
toDownloader.write("X-Have: ${encodeXHave([1], 2)}\r\n\r\n".bytes)
toDownloader.flush()
Thread.sleep(150)
assert performed
assert available.contains(1)
assert thrown == null
}
@Test
public void test200TwoPieces1Available() {
int pieceSize = FileHasher.getPieceSize(1)
int size = (1 << pieceSize) * 9 + 1
initSession(size)
Set<String> headers = readAllHeaders(fromDownloader)
def matcher = null
headers.each {
if (it.startsWith("Range"))
matcher = (it =~ /^Range: (\d+)-(\d+)$/)
}
assert matcher.groupCount() > 0
int start = Integer.parseInt(matcher[0][1])
int end = Integer.parseInt(matcher[0][2])
if (start == 0)
fail("inconlcusive")
toDownloader.write("416 don't have it \r\n".bytes)
toDownloader.write("X-Have: ${encodeXHave([0],2)}\r\n\r\n".bytes)
toDownloader.flush()
downloadThread.join()
assert performed
performed = false
assert available.contains(0)
assert thrown == null
// request same session
downloadThread = new Thread( { perform() } as Runnable)
downloadThread.setDaemon(true)
downloadThread.start()
Thread.sleep(150)
headers = readAllHeaders(fromDownloader)
matcher = null
headers.each {
if (it.startsWith("Range"))
matcher = (it =~ /^Range: (\d+)-(\d+)$/)
}
assert matcher.groupCount() > 0
start = Integer.parseInt(matcher[0][1])
end = Integer.parseInt(matcher[0][2])
assert start == 0
}
@Test
public void testXAlt() throws Exception {
Personas personas = new Personas()
def sources = []
def listener = new Object() {
public void onSourceDiscoveredEvent(SourceDiscoveredEvent e) {
sources << e.source
}
}
eventBus.register(SourceDiscoveredEvent.class, listener)
initSession(20)
readAllHeaders(fromDownloader)
toDownloader.write("416 don't have it\r\n".bytes)
toDownloader.write("X-Alt: ${personas.persona1.toBase64()},${personas.persona2.toBase64()}\r\n\r\n".bytes)
toDownloader.flush()
Thread.sleep(150)
assert sources.contains(personas.persona1)
assert sources.contains(personas.persona2)
assert 2 == sources.size()
}
private static Set<String> readAllHeaders(InputStream is) {
Set<String> rv = new HashSet<>()
String header
while((header = readTillRN(is)) != "")
rv.add(header)
rv
}
} }

View File

@@ -34,4 +34,19 @@ class PiecesTest {
pieces.markDownloaded(piece2) pieces.markDownloaded(piece2)
assert pieces.isComplete() assert pieces.isComplete()
} }
@Test
public void testClaimAvailable() {
pieces = new Pieces(2)
int claimed = pieces.claim([0].toSet())
assert claimed == 0
assert -1 == pieces.claim([0].toSet())
}
@Test
public void testClaimNoneAvailable() {
pieces = new Pieces(20)
int claimed = pieces.claim()
assert -1 == pieces.claim([claimed].toSet())
}
} }

View File

@@ -83,4 +83,11 @@ class SearchIndexTest {
assert found.size() == 1 assert found.size() == 1
assert found.contains("b c.d") assert found.contains("b c.d")
} }
@Test
void testDuplicateTerm() {
initIndex(["MuWire-0.3.3.jar"])
def found = index.search(["muwire", "0", "3", "jar"])
assert found.size() == 1
}
} }

View File

@@ -1,5 +1,5 @@
group = com.muwire group = com.muwire
version = 0.3.0 version = 0.3.6
groovyVersion = 2.4.15 groovyVersion = 2.4.15
slf4jVersion = 1.7.25 slf4jVersion = 1.7.25
spockVersion = 1.1-groovy-2.4 spockVersion = 1.1-groovy-2.4

View File

@@ -17,6 +17,8 @@ import com.muwire.core.Core
import com.muwire.core.download.DownloadStartedEvent import com.muwire.core.download.DownloadStartedEvent
import com.muwire.core.download.UIDownloadCancelledEvent import com.muwire.core.download.UIDownloadCancelledEvent
import com.muwire.core.download.UIDownloadEvent import com.muwire.core.download.UIDownloadEvent
import com.muwire.core.download.UIDownloadPausedEvent
import com.muwire.core.download.UIDownloadResumedEvent
import com.muwire.core.search.QueryEvent import com.muwire.core.search.QueryEvent
import com.muwire.core.search.SearchEvent import com.muwire.core.search.SearchEvent
import com.muwire.core.trust.TrustEvent import com.muwire.core.trust.TrustEvent
@@ -67,7 +69,9 @@ class MainFrameController {
// this can be improved a lot // this can be improved a lot
def replaced = search.toLowerCase().trim().replaceAll(Constants.SPLIT_PATTERN, " ") def replaced = search.toLowerCase().trim().replaceAll(Constants.SPLIT_PATTERN, " ")
def terms = replaced.split(" ") def terms = replaced.split(" ")
searchEvent = new SearchEvent(searchTerms : terms, uuid : uuid, oobInfohash: true) def nonEmpty = []
terms.each { if (it.length() > 0) nonEmpty << it }
searchEvent = new SearchEvent(searchTerms : nonEmpty, uuid : uuid, oobInfohash: true)
} }
core.eventBus.publish(new QueryEvent(searchEvent : searchEvent, firstHop : true, core.eventBus.publish(new QueryEvent(searchEvent : searchEvent, firstHop : true,
replyTo: core.me.destination, receivedOn: core.me.destination, replyTo: core.me.destination, receivedOn: core.me.destination,
@@ -162,6 +166,14 @@ class MainFrameController {
void resume() { void resume() {
def downloader = model.downloads[selectedDownload()].downloader def downloader = model.downloads[selectedDownload()].downloader
downloader.resume() downloader.resume()
core.eventBus.publish(new UIDownloadResumedEvent())
}
@ControllerAction
void pause() {
def downloader = model.downloads[selectedDownload()].downloader
downloader.pause()
core.eventBus.publish(new UIDownloadPausedEvent())
} }
private void markTrust(String tableName, TrustLevel level, def list) { private void markTrust(String tableName, TrustLevel level, def list) {

View File

@@ -77,9 +77,9 @@ class OptionsController {
model.font = text model.font = text
uiSettings.font = text uiSettings.font = text
boolean showMonitor = view.monitorCheckbox.model.isSelected() // boolean showMonitor = view.monitorCheckbox.model.isSelected()
model.showMonitor = showMonitor // model.showMonitor = showMonitor
uiSettings.showMonitor = showMonitor // uiSettings.showMonitor = showMonitor
boolean clearCancelledDownloads = view.clearCancelledDownloadsCheckbox.model.isSelected() boolean clearCancelledDownloads = view.clearCancelledDownloadsCheckbox.model.isSelected()
model.clearCancelledDownloads = clearCancelledDownloads model.clearCancelledDownloads = clearCancelledDownloads
@@ -93,9 +93,9 @@ class OptionsController {
model.excludeLocalResult = excludeLocalResult model.excludeLocalResult = excludeLocalResult
uiSettings.excludeLocalResult = excludeLocalResult uiSettings.excludeLocalResult = excludeLocalResult
boolean showSearchHashes = view.showSearchHashesCheckbox.model.isSelected() // boolean showSearchHashes = view.showSearchHashesCheckbox.model.isSelected()
model.showSearchHashes = showSearchHashes // model.showSearchHashes = showSearchHashes
uiSettings.showSearchHashes = showSearchHashes // uiSettings.showSearchHashes = showSearchHashes
File uiSettingsFile = new File(core.home, "gui.properties") File uiSettingsFile = new File(core.home, "gui.properties")
uiSettingsFile.withOutputStream { uiSettingsFile.withOutputStream {

View File

@@ -98,6 +98,9 @@ class Ready extends AbstractLifecycleHandler {
"Can't connect to I2P router", JOptionPane.WARNING_MESSAGE) "Can't connect to I2P router", JOptionPane.WARNING_MESSAGE)
System.exit(0) System.exit(0)
} }
Runtime.getRuntime().addShutdownHook({
core.shutdown()
})
core.startServices() core.startServices()
application.context.put("muwire-settings", props) application.context.put("muwire-settings", props)
application.context.put("core",core) application.context.put("core",core)

View File

@@ -11,6 +11,7 @@ import com.muwire.core.Core
import com.muwire.core.InfoHash import com.muwire.core.InfoHash
import com.muwire.core.MuWireSettings import com.muwire.core.MuWireSettings
import com.muwire.core.Persona import com.muwire.core.Persona
import com.muwire.core.RouterDisconnectedEvent
import com.muwire.core.connection.ConnectionAttemptStatus import com.muwire.core.connection.ConnectionAttemptStatus
import com.muwire.core.connection.ConnectionEvent import com.muwire.core.connection.ConnectionEvent
import com.muwire.core.connection.DisconnectionEvent import com.muwire.core.connection.DisconnectionEvent
@@ -67,6 +68,8 @@ class MainFrameModel {
@Observable boolean trustButtonsEnabled @Observable boolean trustButtonsEnabled
@Observable boolean cancelButtonEnabled @Observable boolean cancelButtonEnabled
@Observable boolean retryButtonEnabled @Observable boolean retryButtonEnabled
@Observable boolean pauseButtonEnabled
@Observable String resumeButtonText
private final Set<InfoHash> infoHashes = new HashSet<>() private final Set<InfoHash> infoHashes = new HashSet<>()
@@ -135,8 +138,11 @@ class MainFrameModel {
core.eventBus.register(UpdateAvailableEvent.class, this) core.eventBus.register(UpdateAvailableEvent.class, this)
core.eventBus.register(FileDownloadedEvent.class, this) core.eventBus.register(FileDownloadedEvent.class, this)
core.eventBus.register(FileUnsharedEvent.class, this) core.eventBus.register(FileUnsharedEvent.class, this)
core.eventBus.register(RouterDisconnectedEvent.class, this)
timer.schedule({ timer.schedule({
if (core.shutdown.get())
return
int retryInterval = core.muOptions.downloadRetryInterval int retryInterval = core.muOptions.downloadRetryInterval
if (retryInterval > 0) { if (retryInterval > 0) {
retryInterval *= 60000 retryInterval *= 60000
@@ -164,6 +170,8 @@ class MainFrameModel {
watched.addAll(core.muOptions.watchedDirectories) watched.addAll(core.muOptions.watchedDirectories)
builder.getVariable("watched-directories-table").model.fireTableDataChanged() builder.getVariable("watched-directories-table").model.fireTableDataChanged()
watched.each { core.eventBus.publish(new FileSharedEvent(file : new File(it))) } watched.each { core.eventBus.publish(new FileSharedEvent(file : new File(it))) }
resumeButtonText = "Retry"
} }
}) })
@@ -337,6 +345,14 @@ class MainFrameModel {
} }
} }
void onRouterDisconnectedEvent(RouterDisconnectedEvent e) {
runInsideUIAsync {
JOptionPane.showMessageDialog(null, "MuWire lost connection to the I2P router and will now exit.",
"Connection to I2P router lost", JOptionPane.WARNING_MESSAGE)
System.exit(0)
}
}
void onFileDownloadedEvent(FileDownloadedEvent e) { void onFileDownloadedEvent(FileDownloadedEvent e) {
if (!core.muOptions.shareDownloadedFiles) if (!core.muOptions.shareDownloadedFiles)
return return

View File

@@ -135,8 +135,9 @@ class MainFrameView {
} }
} }
panel (constraints : BorderLayout.SOUTH) { panel (constraints : BorderLayout.SOUTH) {
button(text: "Pause", enabled : bind {model.pauseButtonEnabled}, pauseAction)
button(text: "Cancel", enabled : bind {model.cancelButtonEnabled }, cancelAction ) button(text: "Cancel", enabled : bind {model.cancelButtonEnabled }, cancelAction )
button(text: "Retry", enabled : bind {model.retryButtonEnabled}, resumeAction) button(text: bind { model.resumeButtonText }, enabled : bind {model.retryButtonEnabled}, resumeAction)
} }
} }
} }
@@ -295,6 +296,7 @@ class MainFrameView {
if (selectedRow < 0) { if (selectedRow < 0) {
model.cancelButtonEnabled = false model.cancelButtonEnabled = false
model.retryButtonEnabled = false model.retryButtonEnabled = false
model.pauseButtonEnabled = false
return return
} }
def downloader = model.downloads[selectedRow]?.downloader def downloader = model.downloads[selectedRow]?.downloader
@@ -305,15 +307,25 @@ class MainFrameView {
case Downloader.DownloadState.DOWNLOADING : case Downloader.DownloadState.DOWNLOADING :
case Downloader.DownloadState.HASHLIST: case Downloader.DownloadState.HASHLIST:
model.cancelButtonEnabled = true model.cancelButtonEnabled = true
model.pauseButtonEnabled = true
model.retryButtonEnabled = false model.retryButtonEnabled = false
break break
case Downloader.DownloadState.FAILED: case Downloader.DownloadState.FAILED:
model.cancelButtonEnabled = true model.cancelButtonEnabled = true
model.retryButtonEnabled = true model.retryButtonEnabled = true
model.resumeButtonText = "Retry"
model.pauseButtonEnabled = false
break
case Downloader.DownloadState.PAUSED:
model.cancelButtonEnabled = true
model.retryButtonEnabled = true
model.resumeButtonText = "Resume"
model.pauseButtonEnabled = false
break break
default: default:
model.cancelButtonEnabled = false model.cancelButtonEnabled = false
model.retryButtonEnabled = false model.retryButtonEnabled = false
model.pauseButtonEnabled = false
} }
}) })
@@ -421,21 +433,32 @@ class MainFrameView {
int selected = selectedDownloaderRow() int selected = selectedDownloaderRow()
if (selected < 0) if (selected < 0)
return return
boolean pauseEnabled = false
boolean cancelEnabled = false boolean cancelEnabled = false
boolean retryEnabled = false boolean retryEnabled = false
String resumeText = "Retry"
Downloader downloader = model.downloads[selected].downloader Downloader downloader = model.downloads[selected].downloader
switch(downloader.currentState) { switch(downloader.currentState) {
case Downloader.DownloadState.DOWNLOADING: case Downloader.DownloadState.DOWNLOADING:
case Downloader.DownloadState.HASHLIST: case Downloader.DownloadState.HASHLIST:
case Downloader.DownloadState.CONNECTING: case Downloader.DownloadState.CONNECTING:
pauseEnabled = true
cancelEnabled = true cancelEnabled = true
retryEnabled = false retryEnabled = false
break break
case Downloader.DownloadState.FAILED: case Downloader.DownloadState.FAILED:
pauseEnabled = false
cancelEnabled = true cancelEnabled = true
retryEnabled = true retryEnabled = true
break break
case Downloader.DownloadState.PAUSED:
pauseEnabled = false
cancelEnabled = true
retryEnabled = true
resumeText = "Resume"
break
default : default :
pauseEnabled = false
cancelEnabled = false cancelEnabled = false
retryEnabled = false retryEnabled = false
} }
@@ -450,6 +473,12 @@ class MainFrameView {
}) })
menu.add(copyHashToClipboard) menu.add(copyHashToClipboard)
if (pauseEnabled) {
JMenuItem pause = new JMenuItem("Pause")
pause.addActionListener({mvcGroup.controller.pause()})
menu.add(pause)
}
if (cancelEnabled) { if (cancelEnabled) {
JMenuItem cancel = new JMenuItem("Cancel") JMenuItem cancel = new JMenuItem("Cancel")
cancel.addActionListener({mvcGroup.controller.cancel()}) cancel.addActionListener({mvcGroup.controller.cancel()})
@@ -457,7 +486,7 @@ class MainFrameView {
} }
if (retryEnabled) { if (retryEnabled) {
JMenuItem retry = new JMenuItem("Retry") JMenuItem retry = new JMenuItem(resumeText)
retry.addActionListener({mvcGroup.controller.resume()}) retry.addActionListener({mvcGroup.controller.resume()})
menu.add(retry) menu.add(retry)
} }

View File

@@ -89,16 +89,16 @@ class OptionsView {
lnfField = textField(text : bind {model.lnf}, columns : 4, constraints : gbc(gridx : 1, gridy : 1)) lnfField = textField(text : bind {model.lnf}, columns : 4, constraints : gbc(gridx : 1, gridy : 1))
label(text : "Font", constraints : gbc(gridx: 0, gridy : 2)) label(text : "Font", constraints : gbc(gridx: 0, gridy : 2))
fontField = textField(text : bind {model.font}, columns : 4, constraints : gbc(gridx : 1, gridy:2)) fontField = textField(text : bind {model.font}, columns : 4, constraints : gbc(gridx : 1, gridy:2))
label(text : "Show Monitor", constraints : gbc(gridx :0, gridy: 3)) // label(text : "Show Monitor", constraints : gbc(gridx :0, gridy: 3))
monitorCheckbox = checkBox(selected : bind {model.showMonitor}, constraints : gbc(gridx : 1, gridy: 3)) // monitorCheckbox = checkBox(selected : bind {model.showMonitor}, constraints : gbc(gridx : 1, gridy: 3))
label(text : "Clear Cancelled Downloads", constraints: gbc(gridx: 0, gridy:4)) label(text : "Clear Cancelled Downloads", constraints: gbc(gridx: 0, gridy:4))
clearCancelledDownloadsCheckbox = checkBox(selected : bind {model.clearCancelledDownloads}, constraints : gbc(gridx : 1, gridy:4)) clearCancelledDownloadsCheckbox = checkBox(selected : bind {model.clearCancelledDownloads}, constraints : gbc(gridx : 1, gridy:4))
label(text : "Clear Finished Downloads", constraints: gbc(gridx: 0, gridy:5)) label(text : "Clear Finished Downloads", constraints: gbc(gridx: 0, gridy:5))
clearFinishedDownloadsCheckbox = checkBox(selected : bind {model.clearFinishedDownloads}, constraints : gbc(gridx : 1, gridy:5)) clearFinishedDownloadsCheckbox = checkBox(selected : bind {model.clearFinishedDownloads}, constraints : gbc(gridx : 1, gridy:5))
label(text : "Exclude Local Files From Results", constraints: gbc(gridx:0, gridy:6)) label(text : "Exclude Local Files From Results", constraints: gbc(gridx:0, gridy:6))
excludeLocalResultCheckbox = checkBox(selected : bind {model.excludeLocalResult}, constraints : gbc(gridx: 1, gridy : 6)) excludeLocalResultCheckbox = checkBox(selected : bind {model.excludeLocalResult}, constraints : gbc(gridx: 1, gridy : 6))
label(text : "Show Hash Searches In Monitor", constraints: gbc(gridx:0, gridy:7)) // label(text : "Show Hash Searches In Monitor", constraints: gbc(gridx:0, gridy:7))
showSearchHashesCheckbox = checkBox(selected : bind {model.showSearchHashes}, constraints : gbc(gridx: 1, gridy: 7)) // showSearchHashesCheckbox = checkBox(selected : bind {model.showSearchHashes}, constraints : gbc(gridx: 1, gridy: 7))
} }
buttonsPanel = builder.panel { buttonsPanel = builder.panel {
gridBagLayout() gridBagLayout()

View File

@@ -12,12 +12,12 @@ class UISettings {
UISettings(Properties props) { UISettings(Properties props) {
lnf = props.getProperty("lnf", "system") lnf = props.getProperty("lnf", "system")
showMonitor = Boolean.parseBoolean(props.getProperty("showMonitor", "true")) showMonitor = Boolean.parseBoolean(props.getProperty("showMonitor", "false"))
font = props.getProperty("font",null) font = props.getProperty("font",null)
clearCancelledDownloads = Boolean.parseBoolean(props.getProperty("clearCancelledDownloads","false")) clearCancelledDownloads = Boolean.parseBoolean(props.getProperty("clearCancelledDownloads","true"))
clearFinishedDownloads = Boolean.parseBoolean(props.getProperty("clearFinishedDownloads","false")) clearFinishedDownloads = Boolean.parseBoolean(props.getProperty("clearFinishedDownloads","false"))
excludeLocalResult = Boolean.parseBoolean(props.getProperty("excludeLocalResult","false")) excludeLocalResult = Boolean.parseBoolean(props.getProperty("excludeLocalResult","true"))
showSearchHashes = Boolean.parseBoolean(props.getProperty("showSearchHashes","false")) showSearchHashes = Boolean.parseBoolean(props.getProperty("showSearchHashes","true"))
} }
void write(OutputStream out) throws IOException { void write(OutputStream out) throws IOException {

View File

@@ -0,0 +1,61 @@
############################################################
# Default Logging Configuration File
#
# You can use a different file by specifying a filename
# with the java.util.logging.config.file system property.
# For example java -Djava.util.logging.config.file=myfile
############################################################
############################################################
# Global properties
############################################################
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= java.util.logging.FileHandler
# To also add the FileHandler, use the following line instead.
#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= SEVERE
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# default file output is in user's home directory.
java.util.logging.FileHandler.pattern = MuWire.log
java.util.logging.FileHandler.limit = 50000000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
# Example to customize the SimpleFormatter output format
# to print one-line log message like this:
# <level>: <log message> [<date/time>]
#
#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
com.xyz.foo.level = SEVERE

View File

@@ -0,0 +1,61 @@
############################################################
# Default Logging Configuration File
#
# You can use a different file by specifying a filename
# with the java.util.logging.config.file system property.
# For example java -Djava.util.logging.config.file=myfile
############################################################
############################################################
# Global properties
############################################################
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= java.util.logging.FileHandler
# To also add the FileHandler, use the following line instead.
#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= WARNING
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# default file output is in user's home directory.
java.util.logging.FileHandler.pattern = MuWire.log
java.util.logging.FileHandler.limit = 50000000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
# Example to customize the SimpleFormatter output format
# to print one-line log message like this:
# <level>: <log message> [<date/time>]
#
#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
com.xyz.foo.level = SEVERE

View File

@@ -0,0 +1,61 @@
############################################################
# Default Logging Configuration File
#
# You can use a different file by specifying a filename
# with the java.util.logging.config.file system property.
# For example java -Djava.util.logging.config.file=myfile
############################################################
############################################################
# Global properties
############################################################
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= java.util.logging.FileHandler
# To also add the FileHandler, use the following line instead.
#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= INFO
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# default file output is in user's home directory.
java.util.logging.FileHandler.pattern = MuWire.log
java.util.logging.FileHandler.limit = 50000000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
# Example to customize the SimpleFormatter output format
# to print one-line log message like this:
# <level>: <log message> [<date/time>]
#
#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
com.xyz.foo.level = SEVERE

View File

@@ -0,0 +1,62 @@
############################################################
# Default Logging Configuration File
#
# You can use a different file by specifying a filename
# with the java.util.logging.config.file system property.
# For example java -Djava.util.logging.config.file=myfile
############################################################
############################################################
# Global properties
############################################################
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= java.util.logging.FileHandler
# To also add the FileHandler, use the following line instead.
#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= SEVERE
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# default file output is in user's home directory.
java.util.logging.FileHandler.pattern = MuWire.log
java.util.logging.FileHandler.limit = 50000000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
# Example to customize the SimpleFormatter output format
# to print one-line log message like this:
# <level>: <log message> [<date/time>]
#
#java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$s %2$s %5$s %6$s %n
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
com.xyz.foo.level = SEVERE
com.muwire.core.level = FINE