rob
/
torrents.csv
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Adding to README and adding a line to show where server is located

This commit is contained in:
Dessalines 2019-02-06 10:56:46 -08:00
parent 8da2ad592f
commit 3c73736f3c
2 changed files with 21 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
README.md
View File

@ -4,11 +4,13 @@
[Demo Server](https://torrents-csv.ml) [Demo Server](https://torrents-csv.ml)
`Torrents.csv` is a *collaborative* repository of torrents, consisting of a single, searchable `torrents.csv` file. Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites. It comes with a self-hostable [Torrents.csv webserver](https://torrents-csv.ml), a command line search, and a folder scanner to add torrents. `Torrents.csv` is a *collaborative* repository of torrents and their files, consisting of a searchable `torrents.csv`, and `torrent_files.json`. With it you can search for torrents, or files within torrents. It aims to be a universal file system for popular data.
Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites. It comes with a self-hostable [Torrents.csv webserver](https://torrents-csv.ml), a command line search, and a folder scanner to add torrents, and their files.
`Torrents.csv` will only store torrents with at least one seeder to keep the file small, will be periodically purged of non-seeded torrents, and sorted by infohash. `Torrents.csv` will only store torrents with at least one seeder to keep the file small, will be periodically purged of non-seeded torrents, and sorted by infohash.
![img](https://i.imgur.com/GJQE3Xi.png) ![img](https://i.imgur.com/yTFuwpv.png)
To request more torrents, or add your own, go [here](https://gitlab.com/dessalines/torrents.csv/issues). To request more torrents, or add your own, go [here](https://gitlab.com/dessalines/torrents.csv/issues).
@ -61,7 +63,7 @@ bleh season 1 (1993-)
## Uploading / Adding Torrents from a Directory ## Uploading / Adding Torrents from a Directory
An *upload*, consists of making a pull request after running the `scan_torrents.sh` script, which adds torrents from a directory you choose to the `.csv` file, after checking that they aren't already there, and that they have seeders. An *upload*, consists of making a pull request after running the `scan_torrents.sh` script, which adds torrents from a directory you choose to the `.csv` file, after checking that they aren't already there, and that they have seeders. It also adds their files to `torrent_files.json`.
### Requirements ### Requirements
@ -86,7 +88,7 @@ Then [click here](https://gitlab.com/dessalines/torrents.csv/merge_requests/new)
## Web scraping torrents ## Web scraping torrents
`Torrents.csv` has a `Rust` repository for scraping new and top torrents from some torrent sites in the `new_torrents_fetcher` folder. It currently only has skytorrents, but more will be added later. `Torrents.csv` has a `Rust` repository for scraping new and top torrents from some torrent sites in the `new_torrents_fetcher` folder. It currently scrapes skytorrents, magnetdl, and leetx.
### Requirements ### Requirements
@ -107,7 +109,17 @@ infohash;name;size_bytes;created_unix;seeders;leechers;completed;scraped_date
# torrents here... # torrents here...
``` ```
## Sources for new torrents ## How the torrent_files.json looks
- https://www.skytorrents.lol/top100
- https://1337x.to/top-100 ```
- https://1337x.to/trending {
"012ae083ec82bf911f4fe503b9f6df1effaad9ac": [
{
"i": 0, // the index
"p": "File 1", // the path
"l": 88546036A // the size in bytes
},
...
]
}
```

1
server/service/src/main.rs
View File

@ -13,6 +13,7 @@ use std::ops::Deref;
use rusqlite::{Connection, NO_PARAMS}; use rusqlite::{Connection, NO_PARAMS};
fn main() { fn main() {
println!("Access me at http://localhost:8080");
server::new(|| { server::new(|| {
App::new() App::new()
.route("/service/search", http::Method::GET, search) .route("/service/search", http::Method::GET, search)