Updating readme

README.md

@@ -2,41 +2,21 @@
 
 <!-- Torrents.csv - An open source, collaborative repository of torrents, with a self-hostable web server.  -->
 
-`Torrents.csv` is a collaborative, *vetted* repository of torrents, consisting of a single, searchable `torrents.csv` file. Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites via a rust script. 
+[Demo Server](https://torrents-csv.ml)
 
-`Torrents.csv` will only store torrents with at least one seeder to keep the file small, and will be periodically purged of non-seeded torrents, and sorted by seeders descending.
+`Torrents.csv` is a *collaborative* repository of torrents, consisting of a single, searchable `torrents.csv` file. Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites. It comes with a self-hostable [Torrents.csv webserver](https://torrents-csv.ml), a command line search, and a folder scanner to add torrents.
 
-It also comes with a self-hostable [Torrents.csv webserver](https://torrents-csv.ml)
+`Torrents.csv` will only store torrents with at least one seeder to keep the file small, will be periodically purged of non-seeded torrents, and sorted by seeders descending.
 
-![img](https://i.imgur.com/qVmSVMC.png)
+![img](https://i.imgur.com/GJQE3Xi.png)
 
-To request more torrents, or add your own to the file, submit a pull request here.
+To request more torrents, or add your own, go [here](https://gitlab.com/dessalines/torrents.csv/issues).
 
 Made with [Rust](https://www.rust-lang.org), [ripgrep](https://github.com/BurntSushi/ripgrep), [Actix](https://actix.rs/), [Inferno](https://www.infernojs.org), [Typescript](https://www.typescriptlang.org/).
 
-## Searching
+## Webserver
 
-### Requirements
+`Torrents.csv` comes with a simple webserver. [Demo Server](https://torrents-csv.ml)
-
-- [ripgrep](https://github.com/BurntSushi/ripgrep)
-
-### Running
-
-```
-git clone https://gitlab.com/dessalines/torrents.csv
-cd torrents.csv
-./search.sh "bleh season 1"
-```
-
-```
-...
-bleh season 1 (1993-)
-	seeders: 33
-	size: 13GiB
-	link: magnet:?xt=urn:btih:INFO_HASH_HERE
-```
-
-## Running the webserver
 
 ### Requirements
 
@@ -45,11 +25,33 @@ bleh season 1 (1993-)
 
 ### Running
 
-`Torrents.csv` comes with a simple webserver. Run `cd scripts && ./webserver.sh`, and goto http://localhost:8080
+```
+git clone https://gitlab.com/dessalines/torrents.csv
+cd torrents.csv/scripts
+./webserver.sh
+```
+and goto http://localhost:8080
 
 If running on a different host, open up `server/ui/src/env.ts` and change your hostname.
 
-## Uploading / Adding Torrents
+## Command Line Searching
+
+### Requirements
+
+- [ripgrep](https://github.com/BurntSushi/ripgrep)
+
+### Running
+```
+git clone https://gitlab.com/dessalines/torrents.csv
+cd torrents.csv
+./search.sh "bleh season 1"
+bleh season 1 (1993-)
+	seeders: 33
+	size: 13GiB
+	link: magnet:?xt=urn:btih:INFO_HASH_HERE
+```
+
+## Uploading / Adding Torrents from a Directory
 
 An *upload*, consists of making a pull request after running the `scan_torrents.sh` script, which adds torrents from a directory you choose to the `.csv` file, after checking that they aren't already there, and that they have seeders.
 
@@ -66,13 +68,29 @@ An *upload*, consists of making a pull request after running the `scan_torrents.
 git clone https://gitlab.com/[MY_USER]/torrents.csv
 cd torrents.csv/scripts
 ./scan_torrents.sh MY_TORRENTS_DIR # `MY_TORRENTS_DIR` is `~/.local/share/data/qBittorrent/BT_backup/` for qBittorrent on linux, but you can search for where your torrents are stored for your client.
-git commit -am "Adding my torrents"
+git commit -am "Adding [MY_USER] torrents"
 git push
 ```
 
 Then [click here](https://gitlab.com/dessalines/torrents.csv/merge_requests/new) to do a pull/merge request to my branch.
 
-## How the file looks
+## Web scraping torrents
+
+`Torrents.csv` has a `Rust` repository for scraping new and top torrents from some torrent sites in the `new_torrents_fetcher` folder. It currently only has skytorrents, but more will be added later.
+
+### Requirements
+
+- Rust
+
+### Running
+
+```
+git clone https://gitlab.com/dessalines/torrents.csv
+cd torrents.csv/scripts
+./update.sh
+```
+
+## How the .csv file looks
 
 ```sh
 infohash;name;size_bytes;created_unix;seeders;leechers;completed;scraped_date
@@ -81,9 +99,7 @@ infohash;name;size_bytes;created_unix;seeders;leechers;completed;scraped_date
 
 Made with [Rust](https://www.rust-lang.org), [ripgrep](https://github.com/BurntSushi/ripgrep), [Actix](https://actix.rs/), [Inferno](https://www.infernojs.org), and [Typescript](https://www.typescriptlang.org/).
 
-
+## Sources for new torrents
-## Potential sources for new torrents
-
 - https://www.skytorrents.lol/top100
 - https://1337x.to/top-100
 - https://1337x.to/trending

scripts/scrape.sh

@@ -1,13 +0,0 @@
-    
-    
-        found_line=$(rg -n $infohash $torrents_csv | cut -d : -f 1) 
-
-    if [ ! -z $found_line ]; then
-      sed -i "$found_line c$add_line" $torrents_csv
-      echo -e "Found $name, updating peers"
-    else
-      # Append the add lines to the torrents.csv file
-      echo -e "\n$add_line" >> $torrents_csv
-      truncate -s -1 $torrents_csv # Removing last newline
-      echo -e "Added $name"
-    fi

server/ui/src/components/home.tsx

@@ -15,9 +15,9 @@ export class Home extends Component<any, any> {
     return (
       <div>
         <br />
-        <a href={site}>Torrents.csv</a> is a collaborative, <b>vetted</b> git repository of torrents, consisting of a single, searchable <code>torrents.csv</code> file. Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites via a rust script.<br></br><br></br>
+        <a href={site}>Torrents.csv</a> is a <i>collaborative</i> git repository of torrents, consisting of a single, searchable <code>torrents.csv</code> file. Its initially populated with a January 2017 backup of the pirate bay, and new torrents are periodically added from various torrents sites. It comes with a self-hostable webserver, a command line search, and a folder scanner to add torrents.<br /><br />
-        <a href={site}>Torrents.csv</a> will only store torrents with at least one seeder to keep the file small, and will be periodically purged of non-seeded torrents, and sorted by seeders descending.<br></br><br></br>
+        <a href={site}>Torrents.csv</a> will only store torrents with at least one seeder to keep the file small, will be periodically purged of non-seeded torrents, and sorted by seeders descending.<br /><br />
-        To request more torrents, or add your own to the file, go <a href={site}>here</a>.<br></br><br></br>
+        To request more torrents, or add your own, go <a href={site}>here</a>.<br /><br />
         Made with <a href="https://www.rust-lang.org">Rust</a>, <a href="https://github.com/BurntSushi/ripgrep">ripgrep</a>, <a href="https://actix.rs/">Actix</a>, <a href="https://www.infernojs.org">Inferno</a>, and <a href="https://www.typescriptlang.org/">Typescript</a>.
 
       </div>