Compare commits

...

49 Commits

Author SHA1 Message Date
blek b890a9efcb
close off internal resourceD api to WAN 2023-11-16 20:18:18 +10:00
blek 47472eb1c6
implement /resourced/internal/check_resource 2023-11-16 20:14:40 +10:00
blek 0793079cad
add method to check resources in the resourced spec 2023-11-16 19:30:25 +10:00
blek d018c0c2ba
print warning in resourceD if no resources are specified and report number of resources on startup 2023-11-16 19:03:51 +10:00
blek 6e500cf07b add animation to the header 2023-11-11 09:28:01 +01:00
blek 2efdeabf77 add html-minifier to rust dev dockerfile 2023-11-11 09:28:01 +01:00
blek 64edbd48d5 improve dragndrop zone size handling on mobile 2023-11-11 09:28:01 +01:00
blek 3c30f964f0 alternative display for header in mobile mode 2023-11-11 09:28:01 +01:00
blek f976c85e48 print a warning if resourced is disabled 2023-11-11 09:09:48 +01:00
blek cdcecad84d deploy resourced to production dockerfile
Note: I did not test the compose file on my machine before submitting the commit. May or may not work.
2023-11-11 09:09:48 +01:00
blek edf0591bf0 deploy resourceD within dev compose file 2023-11-11 09:09:48 +01:00
blek 24d1318c9f add air to resourceD
more info: https://github.com/cosmtrek/air
2023-11-11 09:09:48 +01:00
blek 75ce4b1311 add swagger reference 2023-11-11 09:09:48 +01:00
blek 15d47c8fea add route to get if the resourced is enabled 2023-11-11 09:09:48 +01:00
blek 8a6c411bf8 fix wrong symbols in the example config 2023-11-11 09:09:48 +01:00
blek e4d3331823 add option to configure min size to cache proxy 2023-11-11 09:09:48 +01:00
blek cc8c9b911a cached proxied resources 2023-11-11 09:09:48 +01:00
blek 915552e62c proxied resources 2023-11-11 09:09:48 +01:00
blek ee450d237a support for http(s):// urls 2023-11-11 09:09:48 +01:00
blek 90d4e8c93f actually serve the resources 2023-11-11 09:09:48 +01:00
blek 4997750fc6 init repo 2023-11-11 09:09:48 +01:00
blek 1aaebdb24d
fix a few problems with template sources 2023-11-06 13:53:08 +10:00
blek 599056a33b
minify templates on compile time 2023-11-06 13:46:05 +10:00
blek a685f5db84
fix drag and drop size 2023-11-03 14:50:02 +10:00
blek 72abd4d5bd
add a ToS switch 2023-11-03 00:08:45 +10:00
blek 57ea39ceda
make code blocks always in one line 2023-11-03 00:02:40 +10:00
blek 575f7d31e8
match colors with the console UI 2023-11-02 23:49:26 +10:00
blek 79ef0634f9
web UI for curlapi/help 2023-11-02 23:44:46 +10:00
blek aca37bc52b
curl API help 2023-11-02 20:00:39 +10:00
blek 164af11b8b
fix compile warning 2023-11-02 19:01:21 +10:00
blek 9ccd3d6212
scratch curl API 2023-11-01 19:19:51 +10:00
blek bae978d1ec
fix wrong .env.example time format 2023-11-01 03:42:14 +10:00
blek 7b607c5ba6
fix typo 2023-11-01 03:11:19 +10:00
blek 6120ce7a30
get rid of function panics in the build.rs 2023-11-01 01:28:00 +10:00
blek 6302d11acd
fix filed dockerfile and use alpine as a base image 2023-11-01 01:26:57 +10:00
blek b1463519d7
fix prod dockerfile build error 2023-11-01 00:46:54 +10:00
blek a0c6c27e8c
bump Cargo.toml version 2023-10-29 19:50:09 +10:00
blek 12614078ad
print version at the footer 2023-10-29 19:44:31 +10:00
blek 99807b9722
fix few obscure errors 2023-10-29 19:25:53 +10:00
blek 32375127a9
display errors 2023-10-29 19:25:33 +10:00
blek 8d5d739568
include branch data 2023-10-29 19:14:58 +10:00
blek b9f0d80dc3
move out code to a function 2023-10-29 19:13:27 +10:00
blek d09f88f7fa
load commit hash at compile time 2023-10-29 19:11:38 +10:00
blek b7b303afb3
remove warp static dir in favor of static_dir! macro 2023-10-29 19:00:54 +10:00
blek ac393b2b7b
add note about extensive configuration 2023-10-29 14:14:50 +10:00
blek 0d02c91626
init: DEPLOYING.md 2023-10-29 14:14:43 +10:00
blek a6d69c319e
Merge remote-tracking branch 'origin/master' into 0.2-dev 2023-10-27 21:29:45 +10:00
blek 8a41d4bef2
add instances list 2023-10-27 10:39:29 +10:00
blek b0cc93c488
remove the indev banner 2023-10-27 10:26:55 +10:00
50 changed files with 1108 additions and 52 deletions

33
DEPLOYING.md Normal file
View File

@ -0,0 +1,33 @@
# Deploying a production instance
Hi fellow sysadmins!
First of all, I want to thank you for using my piece of software.
The instructions can be found below
## Deploying a basic instance
To deploy a basic instance for general public use, follow these simple steps:
1. Clone this repo
2. Copy `docker-compose.prod.yml` to `docker-compose.yml` and edit it to fit your environment
3. Now, there are a few config files that need to be edited by you: `.env`, `filed/.env` and `janitord/.env`. Each directory contains an `.env.example`, and the configuration is pretty straightforward. However, if you are lost check this out: [filed config](#filed-configuration), [janitord config](#janitord-configuration).
4. Configure fileD using `filed/config/filed.toml`. The example is in the same folder. Example contains a lot of self-documenting comments, so it should be pretty simple too.
5. Set `REDIS_PASS` to a secure long string. Not exactly required, but this is something you would want to do
6. Create and start containers with `docker-compose up -d`
7. Route your top level reverse proxy to the `caddy` service or to the port that you opened via the docker compose file.
## More extensive configuration
Well, generally, time-wise, it is not really a good idea to create a custom services configuration.
However, I will guide you through the basic minimal configuration.
Basically, the most minimal blek!File is a fileD service connected to a redis database.
I think that if you are clinically insance, you can set these up as a systemd services or a `screen`ed program.
However, its not really recommended to run this without janitorD as unused files will just clog up your filesystem.
The two requirements for janitorD are to have access to the fileD's usercontent directory and the Redis database.
The default docker configuration mounts `/opt/user_content` to the same volume for both fileD and janitorD.
## FileD configuration
Unless you are running in some kind of super customized docker compose environment, just copying the `.env.example` to `.env` should be enough to get it to run.
Don't forget to set the `REDIS_PASS` to the same value across all services
## JanitorD configuration
Same as [filed config](#filed-configuration), don't forget to set `REDIS_PASS` to a valid value

View File

@ -1,10 +1,3 @@
| ⚠️ This is in a rather early stage of development and shouldn't be deployed |
| --------------------------------------------------------------------------- |
Even though this project is mature enough to be deployed in a public instance,
this is highly discouraged.
However, if you do this, be prepared for [DOS](https://en.wikipedia.org/wiki/Denial-of-service_attack) issues and API changes.
<br/>
<h1 align='center'> <h1 align='center'>
<img src="./filed/static/android-chrome-192x192.png"/> <img src="./filed/static/android-chrome-192x192.png"/>
@ -16,6 +9,21 @@ blek! File is a free service that would help you with file sharing.
The principle is very simple: you upload a file, then download it from another device. The file will be deleted after 1 download or 30 minutes. The principle is very simple: you upload a file, then download it from another device. The file will be deleted after 1 download or 30 minutes.
## Public instances
List of official instances
| Name | Administrator | URL |
| --- | --- | --- |
| 🌠 blek! File | b1ek &lt;me@blek.codes&gt; | [https://file.blek.codes](file.blek.codes) |
To add your instance in this list, fork and open a PR.
To qualify, your instance must be having:
1. Uploads without a password turned on
2. Have proper ToS
3. Come up with a unique name
4. Have a public administrator email
## Licensing ## Licensing
This software is released under GPL3 license, a copyleft license that protects users' freedom by ensuring that all future copies of this software are open source as well. This software is released under GPL3 license, a copyleft license that protects users' freedom by ensuring that all future copies of this software are open source as well.

View File

@ -4,4 +4,11 @@
uri * strip_prefix /qr uri * strip_prefix /qr
reverse_proxy http://qr reverse_proxy http://qr
} }
handle /resource/internal/* {
respond 403
}
handle /resource/* {
uri * strip_prefix /resource
reverse_proxy http://resourced
}
} }

View File

@ -0,0 +1,10 @@
FROM golang:alpine
RUN go install github.com/cosmtrek/air@latest
WORKDIR /opt/code
# The directory will be mounted
# COPY . .
CMD [ "air", "-c", ".air.toml" ]

View File

@ -7,6 +7,6 @@ RUN cargo install cargo-watch && \
RUN apt update && \ RUN apt update && \
apt install nodejs npm -y --no-install-recommends && \ apt install nodejs npm -y --no-install-recommends && \
npm i -g uglify-js npm i -g uglify-js html-minifier
CMD [ "/opt/code/dev-entry.sh" ] CMD [ "/opt/code/dev-entry.sh" ]

View File

@ -7,6 +7,7 @@ services:
networks: networks:
bfile: bfile:
volumes: volumes:
- './.git:/opt/code/.git'
- './filed:/opt/code' - './filed:/opt/code'
- './filed/config:/etc/filed' - './filed/config:/etc/filed'
- '/opt/code/target' - '/opt/code/target'
@ -20,6 +21,15 @@ services:
volumes: volumes:
- './janitor:/opt/code' - './janitor:/opt/code'
- './volatile/files:/opt/user_uploads' - './volatile/files:/opt/user_uploads'
resourced:
build:
context: containers
dockerfile: go-dev.Dockerfile
networks:
bfile:
volumes:
- './resource:/opt/code'
- '/opt/code/tmp'
caddy: caddy:
image: caddy:alpine image: caddy:alpine
volumes: volumes:

View File

@ -2,8 +2,8 @@ version: '3.7'
services: services:
filed: filed:
build: build:
context: filed context: .
dockerfile: Dockerfile.prod dockerfile: filed/Dockerfile.prod
networks: networks:
bfile: bfile:
volumes: volumes:
@ -19,6 +19,15 @@ services:
volumes: volumes:
- './janitor:/config:ro' - './janitor:/config:ro'
- './volatile/files:/opt/user_uploads' - './volatile/files:/opt/user_uploads'
resourced:
build:
context: resource
dockerfile: Dockerfile.prod
networks:
bfile:
pid: host # prefork
volumes:
- './resource:/opt/cont'
caddy: caddy:
image: caddy:alpine image: caddy:alpine
volumes: volumes:

2
filed/Cargo.lock generated
View File

@ -358,7 +358,7 @@ dependencies = [
[[package]] [[package]]
name = "filed" name = "filed"
version = "0.1.0" version = "0.2.0"
dependencies = [ dependencies = [
"argon2", "argon2",
"askama", "askama",

View File

@ -1,6 +1,6 @@
[package] [package]
name = "filed" name = "filed"
version = "0.1.0" version = "0.2.0"
edition = "2021" edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

View File

@ -2,16 +2,17 @@
FROM rust:alpine as builder FROM rust:alpine as builder
WORKDIR /opt/build WORKDIR /opt/build
COPY . . COPY filed .
COPY ./.git ./.git
RUN apk add --no-cache musl-dev upx nodejs yarn && \ RUN apk add --no-cache git musl-dev upx nodejs yarn && \
yarn global add uglify-js yarn global add uglify-js@3.17.4
RUN cargo b -r RUN cargo b -r
RUN strip target/release/filed && upx --best target/release/filed RUN strip target/release/filed && upx --best target/release/filed
# --- deploy --- # --- deploy ---
FROM busybox:musl FROM alpine:3.17
RUN mkdir /config RUN mkdir /config
WORKDIR /config WORKDIR /config

View File

@ -1,5 +1,5 @@
use std::{fs, path::PathBuf, ffi::OsStr, process::Command}; use std::{fs, path::PathBuf, ffi::OsStr, process::Command, error::Error};
use css_minify::optimizations::{Minifier, Level}; use css_minify::optimizations::{Minifier, Level};
@ -9,6 +9,12 @@ fn asset_path(asset: &PathBuf) -> PathBuf {
path path
} }
fn template_path(template: &PathBuf) -> PathBuf {
let mut path = template.components().take(template.components().count() - 2).collect::<PathBuf>();
path.push(template.components().last().unwrap());
path
}
fn extfilter(valid: String, x: Option<&OsStr>) -> bool { fn extfilter(valid: String, x: Option<&OsStr>) -> bool {
if x.is_none() { if x.is_none() {
return false return false
@ -17,9 +23,23 @@ fn extfilter(valid: String, x: Option<&OsStr>) -> bool {
ext == valid ext == valid
} }
fn system(cmd: &str, args: &[&str]) -> Result<String, Box<dyn Error>> {
let out = Command::new(cmd)
.args(args)
.output()
?;
if out.stderr.len() != 0 {
panic!("Got this while running {cmd} with \"{}\": {}", args.join(" "), String::from_utf8(out.stderr).unwrap())
}
Ok(String::from_utf8(out.stdout)?)
}
fn main() { fn main() {
println!("cargo:rerun-if-changed=static/assets"); println!("cargo:rerun-if-changed=static/assets");
println!("cargo:rerun-if-changed=templates/source");
let assets = fs::read_dir("static/assets").unwrap(); let assets = fs::read_dir("static/assets").unwrap();
let assets = assets let assets = assets
@ -52,11 +72,45 @@ fn main() {
scripts.iter().for_each(|asset| { scripts.iter().for_each(|asset| {
Command::new("uglifyjs") Command::new("uglifyjs")
.arg("-c")
.arg(asset) .arg(asset)
.arg("-o") .arg("-o")
.arg(asset_path(asset)) .arg(asset_path(asset))
.arg("-c")
.spawn() .spawn()
.unwrap(); .unwrap();
}) });
// precompile templates
let templates = fs::read_dir("templates/source").unwrap();
let templates = templates
.map(|x| x.unwrap().path().canonicalize().unwrap())
.filter(|x| extfilter("html".into(), x.extension()))
.collect::<Vec<PathBuf>>();
templates.iter().for_each(|template| {
Command::new("html-minifier")
.arg(template.canonicalize().unwrap())
.arg("--collapse-whitespace")
// .arg("--minify-js")
// .arg("--minify-css")
// .arg("--keep-closing-slash")
.arg("--output")
.arg(template_path(template))
.spawn()
.unwrap();
});
let commit = system("git", &["rev-parse", "HEAD"]).map_err(|x| x.to_string());
let branch = system("git", &["rev-parse", "--abbrev-ref", "HEAD"]).map_err(|x| x.to_string());
match commit {
Err(err) => panic!("Can't get commit: {}", err),
Ok(commit) => println!("cargo:rustc-env=COMMIT_HASH={commit}")
}
match branch {
Err(err) => panic!("Can't get commit: {}", err),
Ok(branch) => println!("cargo:rustc-env=COMMIT_BRANCH={branch}")
}
} }

View File

@ -84,3 +84,7 @@ sudo_delete=false
# Whether /api/upload is enabled # Whether /api/upload is enabled
# It is not recommended to enable it if API key auth is not enabled # It is not recommended to enable it if API key auth is not enabled
upload=false upload=false
# Whether curlapi is enabled
# curl {url}/curlapi/help for more info
curlapi=true

View File

@ -5,4 +5,4 @@ cd /opt/code
cargo check cargo check
cargo build cargo build
cargo watch -w src -w templates -w static -x run cargo watch -w src -w templates/source -w static -x run

View File

@ -112,6 +112,10 @@ pub struct APISettings {
/// Whether /api/upload is enabled /// Whether /api/upload is enabled
#[serde(default)] #[serde(default)]
pub upload: bool, pub upload: bool,
/// Whether curlapi is enabled
#[serde(default)]
pub curlapi: bool
} }
impl Default for APISettings { impl Default for APISettings {
@ -123,7 +127,8 @@ impl Default for APISettings {
get_all_own_only: true, get_all_own_only: true,
delete: false, delete: false,
sudo_delete: false, sudo_delete: false,
upload: false upload: false,
curlapi: true
} }
} }
} }

View File

@ -15,6 +15,23 @@ pub struct Redis {
pub prefix: String pub prefix: String
} }
#[derive(Debug, Clone)]
pub struct VersionData {
pub commit: String,
pub short_commit: String,
pub branch: String
}
impl Default for VersionData {
fn default() -> Self {
VersionData {
commit: env!("COMMIT_HASH").to_string(),
short_commit: env!("COMMIT_HASH").to_string().chars().take(6).collect(),
branch: env!("COMMIT_BRANCH").to_string()
}
}
}
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct Env { pub struct Env {
pub logging: bool, pub logging: bool,
@ -24,7 +41,8 @@ pub struct Env {
pub filedir: String, pub filedir: String,
pub instanceurl: String, pub instanceurl: String,
pub uploadspath: String, pub uploadspath: String,
pub confpath: String pub confpath: String,
pub version: VersionData
} }
fn get_var<T: Into<String>, O: From<String>>(name: T) -> Result<O, String> { fn get_var<T: Into<String>, O: From<String>>(name: T) -> Result<O, String> {
@ -140,7 +158,8 @@ pub fn loadenv() -> Result<Env, Box<dyn std::error::Error>> {
return Err(format!("CONF_FILE is {}, which is not a file!", spath).into()) return Err(format!("CONF_FILE is {}, which is not a file!", spath).into())
} }
spath spath
} },
version: VersionData::default()
} }
) )
} }

11
filed/src/web/curlapi.rs Normal file
View File

@ -0,0 +1,11 @@
use warp::{Filter, reply::Reply, reject::Rejection};
use super::state::SharedState;
mod upload;
mod help;
pub fn get_routes(state: SharedState) -> impl Filter<Extract = impl Reply, Error = Rejection> + Clone {
upload::get_routes(state.clone())
.or(help::get_routes(state))
}

View File

@ -0,0 +1,85 @@
use askama::Template;
use warp::{Filter, reply::{Reply, html}, reject::Rejection};
use crate::web::{state::SharedState, pages::CurlHelpPage, rejection::HttpReject};
pub async fn help(state: SharedState, ua: String) -> Result<Box<dyn Reply>, Rejection> {
if ! ua.starts_with("curl/") {
let page = CurlHelpPage { conf: state.config.clone(), env: state.env.clone() };
return Ok(
Box::new(
html(page.render().map_err(|x| HttpReject::AskamaError(x))?)
)
)
}
let brand = format!(
"{} \x1b[1m{}\x1b[0m {}",
state.config.brand.instance_emoji,
state.config.brand.instance_name,
{
if state.config.brand.instance_name != "blek! File" {
"\n\x1b[90mPowered by blek! File\x1b[0m"
} else { "" }
}
);
let mut warns: String = String::new();
if ! state.config.api.curlapi {
warns += "\x1b[1;31mWarning: curl API is disabled on this instance.\nYou can use the web UI to upload files.\x1b[0m\n\n"
}
if ! state.config.files.allow_uploads {
warns += {
format!(
"\x1b[1;31mWarning: all uploads are disabled on this instance{}\x1b[0m",
{
if let Some(reason) = state.config.files.upload_disable_reason {
format!(" for this reason:\n\"{}\"", reason)
} else { ".".to_string() }
}
).as_str()
}
}
let instance = state.env.instanceurl;
let help =
format!(
"To upload a new file, you can POST it like this:
\x1b[32mcurl\x1b[0m \x1b[33m-X POST\x1b[0m \x1b[34m{instance}/curlapi/upload\x1b[0m \x1b[33m-F'file=@file.txt'\x1b[0m \x1b[33m-F'tos_consent=on'\x1b[0m
You can also add a password:
\x1b[32mcurl\x1b[0m \x1b[33m-X POST\x1b[0m \x1b[34m{instance}/curlapi/upload\x1b[0m \x1b[33m-F'file=@file.txt'\x1b[0m \x1b[33m-F'filename=uwu'\x1b[0m \x1b[33m-F'tos_consent=on'\x1b[0m \x1b[33m-F'named=on'\x1b[0m
The `named=on` switch is neede because this API is basically
the HTML used at the regular web UI form wrapped into this URL
\x1b[1;32mIMPORTANT:\x1b[0m Read the terms of service \x1b[1mbefore\x1b[0m uploading the file!
The ToS can be found here: \x1b[34m{instance}/tos\x1b[0m .
{warns}"
);
Ok(
Box::new(
format!("
\x1b[31m \x1b[0m \x1b[35m \x1b[0m
\x1b[32m\x1b[0m \x1b[95m \x1b[0m
\x1b[34m\x1b[0m \x1b[35m \x1b[0m
{brand}
{help}
").to_string())
)
}
pub fn get_routes(state: SharedState) -> impl Filter<Extract = impl Reply, Error = Rejection> + Clone {
warp::any()
.and(warp::path!("curlapi" / "help"))
.and(
warp::any()
.map(move || state.clone())
)
.and(
warp::header::<String>("user-agent")
)
.and_then(help)
}

View File

@ -0,0 +1,139 @@
use std::net::IpAddr;
use warp::{filters::multipart::FormData, reply::{Reply, with_status}, reject::Rejection, Filter};
use warp_real_ip::real_ip;
use crate::{web::{state::SharedState, forms::{FormElement, UploadFormData}, rejection::HttpReject}, files::File};
pub async fn upload(form: FormData, ip: Option<IpAddr>, state: SharedState) -> Result<Box<dyn Reply>, Rejection> {
if ! state.config.files.allow_uploads {
return Ok(
Box::new(
with_status(
match state.config.files.upload_disable_reason {
Some(reason) => format!("Uploads are disabled for the following reason:\n{reason}"),
None => "Uploads are disabled.".into()
},
warp::http::StatusCode::SERVICE_UNAVAILABLE
)
)
)
}
let params = FormElement::from_formdata(form)
.await
.map_err(|x| HttpReject::WarpError(x))?;
if let Some(consent) = params.get("tos_consent") {
if consent.data != "on".bytes().collect::<Vec<u8>>() {
return Ok(
Box::new(
format!("You need to agree to the ToS to upload a file.\nSee {}/curlapi/help for details\n\nTo agree to the ToS, add a -F'tos_consent=on'\n", state.env.instanceurl)
)
)
}
} else {
return Ok(
Box::new(
format!("You need to agree to the ToS to upload a file.\nSee {}/curlapi/help for details\n\nTo agree to the ToS, add a -F'tos_consent=on'\n", state.env.instanceurl)
)
)
}
let formdata = UploadFormData::from_formdata(params, true);
if let Some(formdata) = formdata {
let mut breaks_conf = false;
if (!state.config.files.allow_custom_names) && formdata.filename.is_some() {
breaks_conf = true;
}
if (!state.config.files.allow_pass_protection) && formdata.password.is_some() {
breaks_conf = true;
}
if breaks_conf {
return Ok(
Box::new(
with_status(
"Attempt to set name or password when they are disabled".to_string(),
warp::http::StatusCode::BAD_REQUEST
)
)
);
}
if let Some(pass) = state.config.files.upload_pass {
let pass_valid: bool;
if let Some(upass) = formdata.instancepass {
pass_valid = upass == pass;
} else {
pass_valid = false
}
if ! pass_valid {
return Ok(
Box::new(
with_status(
"Invalid instance password".to_string(),
warp::http::StatusCode::BAD_REQUEST
)
)
);
}
}
let file = File::create(
formdata.file,
formdata.mime,
formdata.filename.clone(),
state.env.clone(),
formdata.delmode,
formdata.password,
ip
).await.map_err(|x| HttpReject::StringError(x.to_string()))?;
state.file_mgr.save(&file, formdata.lookup_kind).map_err(|x| HttpReject::StringError(x.to_string()))?;
return Ok(
Box::new(
format!(
concat!(
"File uploaded successfully.\n",
"It is available via this link:\n\n",
"{}/upload/{}\n"
),
state.env.instanceurl,
urlencoding::encode(
match formdata.filename {
Some(name) => name,
None => file.hash()
}.as_str()
)
)
)
);
} else {
Ok(
Box::new(
with_status(
"Invalid form".to_string(),
warp::http::StatusCode::BAD_REQUEST
)
)
)
}
}
pub fn get_routes(state: SharedState) -> impl Filter<Extract = impl Reply, Error = Rejection> + Clone {
warp::post()
.and(warp::path!("curlapi" / "upload"))
.and(warp::multipart::form())
.and(real_ip(vec![state.env.proxy_addr]))
.and(
warp::any()
.map(move || state.clone())
)
.and_then(upload)
}

View File

@ -17,9 +17,9 @@ use crate::files::{File, lookup::LookupKind, DeleteMode};
use super::{state::SharedState, pages::{UploadSuccessPage, ErrorPage}, rejection::HttpReject}; use super::{state::SharedState, pages::{UploadSuccessPage, ErrorPage}, rejection::HttpReject};
#[derive(Debug, Serialize, Clone)] #[derive(Debug, Serialize, Clone)]
struct FormElement { pub struct FormElement {
data: Vec<u8>, pub data: Vec<u8>,
mime: String pub mime: String
} }
impl FormElement { impl FormElement {
@ -58,15 +58,15 @@ impl FormElement {
} }
} }
struct UploadFormData { pub struct UploadFormData {
filename: Option<String>, pub filename: Option<String>,
password: Option<String>, pub password: Option<String>,
instancepass: Option<String>, pub instancepass: Option<String>,
lookup_kind: LookupKind, pub lookup_kind: LookupKind,
delmode: DeleteMode, pub delmode: DeleteMode,
file: Vec<u8>, pub file: Vec<u8>,
mime: String, pub mime: String,
tos_consent: bool pub tos_consent: bool
} }
impl Default for UploadFormData { impl Default for UploadFormData {
@ -86,7 +86,7 @@ impl Default for UploadFormData {
impl UploadFormData { impl UploadFormData {
pub fn from_formdata(data: HashMap<String, FormElement>) -> Option<UploadFormData> { pub fn from_formdata(data: HashMap<String, FormElement>, use_defaults: bool) -> Option<UploadFormData> {
let mut out = Self::default(); let mut out = Self::default();
// Add a name // Add a name
@ -125,7 +125,9 @@ impl UploadFormData {
} }
}, },
None => { None => {
return None if ! use_defaults {
return None
}
} }
} }
@ -169,7 +171,7 @@ pub async fn upload(form: FormData, ip: Option<IpAddr>, state: SharedState) -> R
} }
let params: HashMap<String, FormElement> = FormElement::from_formdata(form).await.map_err(|x| HttpReject::WarpError(x))?; let params: HashMap<String, FormElement> = FormElement::from_formdata(form).await.map_err(|x| HttpReject::WarpError(x))?;
let formdata = UploadFormData::from_formdata(params.clone()); let formdata = UploadFormData::from_formdata(params.clone(), false);
if let Some(formdata) = formdata { if let Some(formdata) = formdata {

View File

@ -2,32 +2,28 @@
web - The part of filed that handles everything related to HTTP web - The part of filed that handles everything related to HTTP
*/ */
use std::env::current_dir;
use static_dir::static_dir; use static_dir::static_dir;
use warp::{Filter, reply::Reply, reject::Rejection}; use warp::{Filter, reply::Reply, reject::Rejection};
use crate::{env::Env, files::lookup::FileManager, config::types::Config}; use crate::{env::Env, files::lookup::FileManager, config::types::Config};
mod pages; mod pages;
mod forms; pub mod forms;
pub mod state; pub mod state;
mod rejection; mod rejection;
mod api; mod api;
mod uploaded; mod uploaded;
mod curlapi;
use state::SharedState; use state::SharedState;
pub fn routes(state: SharedState) -> impl Filter<Extract = impl Reply, Error = Rejection> + Clone { pub fn routes(state: SharedState) -> impl Filter<Extract = impl Reply, Error = Rejection> + Clone {
let staticpath = current_dir().unwrap(); static_dir!("static")
let staticpath = staticpath.to_str().unwrap().to_string() + "/static"; .or(curlapi::get_routes(state.clone()))
.or(pages::get_routes(state.clone()))
pages::get_routes(state.clone())
.or(forms::get_routes(state.clone())) .or(forms::get_routes(state.clone()))
.or(api::get_routes(state.clone())) .or(api::get_routes(state.clone()))
.or(uploaded::get_uploaded(state)) .or(uploaded::get_uploaded(state))
.or(static_dir!("static"))
.or(warp::fs::dir(staticpath.to_string()))
} }
/* /*

View File

@ -88,6 +88,14 @@ pub struct ErrorPage {
pub link_text: Option<String> pub link_text: Option<String>
} }
#[derive(Template)]
#[template( path = "curlapi_help.html" )]
#[allow(dead_code)]
pub struct CurlHelpPage {
pub env: Env,
pub conf: Config
}
pub async fn uploaded(query: HashMap<String, String>, state: SharedState) -> Result<Html<String>, Rejection> { pub async fn uploaded(query: HashMap<String, String>, state: SharedState) -> Result<Html<String>, Rejection> {
if ! query.contains_key("file") { if ! query.contains_key("file") {

View File

@ -25,3 +25,6 @@
.alert.blue .alert-title { .alert.blue .alert-title {
background: #203050; background: #203050;
} }
.alert.green .alert-title {
background: #205030;
}

View File

@ -0,0 +1,23 @@
.code {
display: block;
padding: 1em;
border: 1px solid var(--header-sec-color);
border-radius: 12px;
overflow-x: auto;
}
.code .inner {
width: max-content;
display: block;
}
.code-inline {
display: inline;
background: #00000010;
border: 1px solid #c2c4c210;
border-radius: 4px;
padding: 2px 4px;
}
.code, .code-inline {
font-family: monospace;
line-height: 1.5em;
}

View File

@ -6,6 +6,27 @@
/** @type {HTMLElement} */ /** @type {HTMLElement} */
const root_drag_rop = document.getElementsByClassName('file-drag-n-drop')[0]; const root_drag_rop = document.getElementsByClassName('file-drag-n-drop')[0];
// make the root drag&drop element an ideal circle
function updateDragNDrop() {
if (document.body.scrollWidth < 667) {
// mobile
delete root_drag_rop.style.width;
delete root_drag_rop.style.height;
return
}
const width = root_drag_rop.offsetWidth;
root_drag_rop.style.width = width + 'px';
root_drag_rop.style.height = width + 'px';
}
updateDragNDrop();
document.onresize = updateDragNDrop();
/** @type {HTMLElement} */ /** @type {HTMLElement} */
const drag_rop = document.getElementsByClassName('file-drag-n-drop-inside')[0]; const drag_rop = document.getElementsByClassName('file-drag-n-drop-inside')[0];
const dr_rop_t = document.getElementsByClassName('file-drag-n-drop-inside-text')[0]; const dr_rop_t = document.getElementsByClassName('file-drag-n-drop-inside-text')[0];

View File

@ -1,7 +1,7 @@
.file-drag-n-drop { .file-drag-n-drop {
display: block; display: block;
cursor: pointer; cursor: pointer;
width: 440px; width: 100%;
height: 440px; height: 440px;
background: var(--header-sec-color); background: var(--header-sec-color);
border-radius: 16px; border-radius: 16px;

2
filed/templates/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
*
!.gitignore

View File

@ -16,6 +16,22 @@
<title>{{ conf.brand.instance_name }}</title> <title>{{ conf.brand.instance_name }}</title>
<style> .footer svg { height: 32px; margin: 10px 0 } </style> <style> .footer svg { height: 32px; margin: 10px 0 } </style>
<style>
.header .header-text, .header .header-home, .header .header-bg { transition: 250ms ease; }
@media (max-width:667px) {
.header .header-text { text-align: center }
.header .header-home { width: 100% }
.header .header-bg { left: 50% !important; transform: translateX(-50%); -webkit-mask-image: linear-gradient(90deg, #0000, #000, #0000); }
{%- if conf.brand.instance_motto.len() != 0 -%}
.header .header-home-motto {
display:none
}
{%- endif -%}
}
</style>
{%- if cfg!(debug_assertions) -%} {%- if cfg!(debug_assertions) -%}
<link rel="stylesheet" href="/alert.css" /> <link rel="stylesheet" href="/alert.css" />
@ -33,7 +49,7 @@
{{- conf.brand.instance_name -}} {{- conf.brand.instance_name -}}
{%- if conf.brand.instance_motto.len() != 0 -%} {%- if conf.brand.instance_motto.len() != 0 -%}
{#- Whitespace control is stupid -#} {#- Whitespace control is stupid -#}
{{- " - " -}}{{- conf.brand.instance_motto -}} <span class="header-home-motto">{{- " - " -}}{{- conf.brand.instance_motto -}}</span>
{%- endif -%} {%- endif -%}
</a> </a>
</div> </div>
@ -67,6 +83,13 @@
<td> <td>
<small>Made with Rust and &lt;3</small> <small>Made with Rust and &lt;3</small>
<small style="display:block">
Version
<a href="https://git.blek.codes/blek/bfile/commit/{{ env.version.commit }}" target="_blank">
{{ env!("CARGO_PKG_VERSION") }} ({{ env.version.branch -}}/{{- env.version.short_commit }})
</a>
</small>
<ul style='margin:10px 0'> <ul style='margin:10px 0'>
<li> <li>
<a href="https://git.blek.codes/blek/bfile"> <a href="https://git.blek.codes/blek/bfile">

View File

@ -0,0 +1 @@
<svg xmlns='http://www.w3.org/2000/svg' fill='currentColor' viewBox='0 0 217.2 80.2'><path stroke-linecap='round' stroke-width='.94' d='M167.5 78.8h-30.6v-1h.1a9.56 9.56 0 0 0 2.6-.34 7.21 7.21 0 0 0 3.2-1.86 7.38 7.38 0 0 0 2.08-4.19 10.34 10.34 0 0 0 .12-1.61v-57a9.37 9.37 0 0 0-.34-2.6 7.18 7.18 0 0 0-1.91-3.2 7.62 7.62 0 0 0-4.42-2.1 10.54 10.54 0 0 0-1.43-.1v-1h10.2q4.71 0 7.38-.94a9.48 9.48 0 0 0 .42-.16q1.74-.77 2.75-1.77a5.19 5.19 0 0 0 .75-.93h1v52.6l17-14.3a28.74 28.74 0 0 0 1.94-1.76q1.86-1.85 2.62-3.47a6.17 6.17 0 0 0 .34-.87 5.92 5.92 0 0 0 .09-.43q.3-1.9-1.08-2.38a2.57 2.57 0 0 0-.31-.09l-1.3-.3v-1h14.9v1l-1 .2a11.7 11.7 0 0 0-2.6.76q-1.12.47-2.28 1.2a19.22 19.22 0 0 0-.07.04q-1.94 1.22-5.2 3.84a159.62 159.62 0 0 0-2.15 1.76l-6.9 5.8 19.9 30.6a33.2 33.2 0 0 0 1.08 1.36q1.06 1.24 1.93 1.97a7.93 7.93 0 0 0 .29.22q1.4 1.05 3.1 1.05v1h-26.4v-1q2.27 0 3.13-.33a1.76 1.76 0 0 0 .17-.07q.74-.25.87-.97a1.86 1.86 0 0 0 .03-.33q0-.81-.57-2.04a11.36 11.36 0 0 0-.13-.26 85.38 85.38 0 0 0-.32-.63l-.3-.59a148.04 148.04 0 0 0-.33-.63 31.93 31.93 0 0 0-.53-.97l-.6-1.02a54.25 54.25 0 0 0-.22-.36l-11.4-18.3-3.7 3.2v15.3a9.56 9.56 0 0 0 .34 2.6 7.21 7.21 0 0 0 1.86 3.2 7.38 7.38 0 0 0 4.19 2.08 10.34 10.34 0 0 0 1.61.12h.1v1ZM9.1 80.2h-1V12.8a9.37 9.37 0 0 0-.34-2.6A7.18 7.18 0 0 0 5.85 7a7.62 7.62 0 0 0-4.42-2.1A10.54 10.54 0 0 0 0 4.8v-1h10.2a49.91 49.91 0 0 0 2.64-.06q3.77-.2 5.44-1.03a4.6 4.6 0 0 0 .02-.01 10.17 10.17 0 0 0 1.48-.9q.81-.6 1.37-1.3a5.73 5.73 0 0 0 .35-.5h1v36.4q2.3-4.5 6.45-7.2t9.15-2.7q5.9 0 10.5 3.5a22.42 22.42 0 0 1 5.7 6.49 28.24 28.24 0 0 1 1.55 3.01q2.65 6 2.65 13.7a36.88 36.88 0 0 1-.7 7.3 28.48 28.48 0 0 1-2.25 6.8q-2.95 6.1-8.25 9.5a21.45 21.45 0 0 1-9.17 3.21 27.24 27.24 0 0 1-3.23.19 21.87 21.87 0 0 1-6.22-.83 14.8 14.8 0 0 1-9.78-9.07l-9.8 9.9Zm128-34.4.1 1.5h-33.6v.5a38.84 38.84 0 0 0 .58 6.93q.7 3.87 2.26 6.98a20.6 20.6 0 0 0 1.86 3.04 15.79 15.79 0 0 0 4.92 4.45q3.36 1.9 7.68 1.9 4.8 0 9.15-2.35 4.35-2.35 7.05-6.35l.9.3q-1.7 5.1-5.3 9-3.6 3.9-8.5 6.1a25.45 25.45 0 0 1-10.07 2.2 29.25 29.25 0 0 1-.53 0q-6.9 0-12.25-3.4a22.86 22.86 0 0 1-7.97-8.7 27.29 27.29 0 0 1-.38-.75 28.67 28.67 0 0 1-2.7-9.03 36.6 36.6 0 0 1-.3-4.82 32.79 32.79 0 0 1 .77-7.28 25.67 25.67 0 0 1 2.53-6.72q3.3-6 9.05-9.35 5.75-3.35 13.05-3.35a26.46 26.46 0 0 1 6.02.66 21.48 21.48 0 0 1 4.83 1.74q4.75 2.4 7.6 6.75a20.08 20.08 0 0 1 3.04 8.13 24.45 24.45 0 0 1 .21 1.92Zm-47.2 33H59.3v-1a9.82 9.82 0 0 0 2.65-.34 7.46 7.46 0 0 0 3.2-1.81 7.12 7.12 0 0 0 2.08-3.94 10.33 10.33 0 0 0 .17-1.91v-57a9.37 9.37 0 0 0-.34-2.6A7.18 7.18 0 0 0 65.15 7a7.62 7.62 0 0 0-4.42-2.1 10.54 10.54 0 0 0-1.43-.1v-1h10.2q4.71 0 7.38-.94a9.48 9.48 0 0 0 .42-.16q1.74-.77 2.75-1.77A5.19 5.19 0 0 0 80.8 0h1v69.8q0 3.7 2.25 5.85 2.25 2.15 5.85 2.15v1ZM22.5 39.9V65a27.97 27.97 0 0 0 .52 3.96q.74 3.5 2.38 5.89 2.5 3.65 6.6 3.65 4.95 0 7.63-5.75a19.3 19.3 0 0 0 .37-.85 28.56 28.56 0 0 0 1.42-4.73Q42.7 61.3 42.7 52.5a97.73 97.73 0 0 0-.13-5.24q-.3-5.58-1.29-9.26a20.42 20.42 0 0 0-.68-2.1q-1.23-3.15-3.27-4.46A6.03 6.03 0 0 0 34 30.5a9.9 9.9 0 0 0-6.95 2.81 12.76 12.76 0 0 0-.1.09 18.73 18.73 0 0 0-3.3 4.1 16.72 16.72 0 0 0-1.15 2.4Zm186.6 11.8h-1a115.54 115.54 0 0 0-.18-6.57q-.29-5.13-1.07-9.13a148.1 148.1 0 0 0-.96-4.55q-.5-2.23-1.04-4.2a78.12 78.12 0 0 0-.65-2.25 142.68 142.68 0 0 0-1.2-3.63q-.58-1.64-1.13-3.08a78.3 78.3 0 0 0-.37-.94 18.06 18.06 0 0 1-1.17-4.4 15.99 15.99 0 0 1-.13-2.05 10.23 10.23 0 0 1 .35-2.75 7.75 7.75 0 0 1 1.95-3.4 7.7 7.7 0 0 1 4.48-2.23 10.83 10.83 0 0 1 1.62-.12 10.48 10.48 0 0 1 2.76.34 7.7 7.7 0 0 1 3.49 2.01 7.91 7.91 0 0 1 2.25 4.67 11 11 0 0 1 .1 1.48 17.47 17.47 0 0 1-.87 5.38 20.4 20.4 0 0 1-.38 1.07q-.86 2.23-1.84 5.01a284.93 284.93 0 0 0-.91 2.64q-1.6 4.6-2.85 11a59.41 59.41 0 0 0-.76 5.2q-.49 4.69-.49 10.5Zm-105.4-5.9H124q0-7.9-2.55-12.8a11.57 11.57 0 0 0-1.65-2.42 7.12 7.12 0 0 0-5.5-2.48 7.41 7.41 0 0 0-5.34 2.27q-.99.97-1.85 2.34a16.41 16.41 0 0 0-.11.19 19.79 19.79 0 0 0-1.94 4.45q-.69 2.22-1.04 4.87a43.13 43.13 0 0 0-.32 3.58Zm98.88 31.98a8.17 8.17 0 0 0 5.92 2.42 9.73 9.73 0 0 0 .1 0 8.18 8.18 0 0 0 5.9-2.5 10.03 10.03 0 0 0 .08-.08A8.17 8.17 0 0 0 217 71.7a10.28 10.28 0 0 0 0-.34 7.88 7.88 0 0 0-2.6-5.76 10.87 10.87 0 0 0-.33-.3 8.38 8.38 0 0 0-5.57-2.1 8.2 8.2 0 0 0-3.22.64 9.25 9.25 0 0 0-2.68 1.76 7.87 7.87 0 0 0-2.03 2.92 8.6 8.6 0 0 0-.57 3.18 9.73 9.73 0 0 0 0 .1 8.18 8.18 0 0 0 2.5 5.9 10.03 10.03 0 0 0 .08.08Z' font-size='12' vector-effect='non-scaling-stroke'/></svg>

After

Width:  |  Height:  |  Size: 4.4 KiB

View File

@ -0,0 +1,113 @@
{% extends "base.html" %}
{% block head %}
<link rel='stylesheet' href="/code.css" />
<link rel='stylesheet' href="/alert.css" />
<link rel='stylesheet' href="/js-only.css" />
<style>
.copy-btn { font-size: 70%; transform: translateY(-25%); display: inline-block }
</style>
{% endblock %}
{% block body %}
<div style="max-width:95vw;width:fit-content;margin:0 auto">
<h1 style="text-align:center">Curl API</h1>
<p>
blek! File has an API for uploading files via cURL.
To upload a file via cURL, follow these instructions:
</p>
<p>
To upload a file, POST it like this:
<a href="#" class="copy-btn" data-clipboard-text="curl -X POST {{env.instanceurl}}/curlapi/upload -F'file=@file.txt' -F'tos_consent=on'">
Copy!
</a>
</p>
<div class='code'>
<span class='inner'>
<span style='color:green'>curl</span>
<span style='color:orange'>-X POST</span>
<span style='color:darkcyan'>{{env.instanceurl}}/curlapi/upload</span>
<span style='color:orange'>-F'file=@file.txt'</span>
<span style='color:orange'>-F'tos_consent=on'</span>
</span>
</div>
<p>
To add a password, do it like this:
<a href="#" class="copy-btn" data-clipboard-text="curl -X POST {{env.instanceurl}}/curlapi/upload -F'file=@file.txt' -F'filename=uwu' -F'tos_consent=on' -F'named=on'">
Copy!
</a>
</p>
<div class='code'>
<span class='inner'>
<span style='color:green'>curl</span>
<span style='color:orange'>-X POST</span>
<span style='color:darkcyan'>{{env.instanceurl}}/curlapi/upload</span>
<span style='color:orange'>-F'file=@file.txt'</span>
<span style='color:orange'>-F'tos_consent=on'</span>
<span style='color:orange'>-F'named=on'</span>
</span>
</div>
<p>
Note that the
<span class='code-inline'>named=on</span>
switch is required.
Its needed because the curl API is basically a wrapper of
<a href="/">this</a>
HTML form.
</p>
<div class="alert green">
<h1 class="alert-title">
Important
</h1>
<p class="alert-text">
Read the
<a href="/tos">Terms of Service</a>
<b>before</b>
uploading a file.
<br/>
You agree to them by adding the
<span class="code-inline">tos_consent=on</span>
switch.
</p>
</div>
<div class='alert blue'>
<h1 class='alert-title'>Web UI</h1>
<div class='alert-text'>
<p>
Hey, it looks like you are viewing this page from a browser!<br/>
You can use the Web UI as well to upload a file!
</p>
<p style='margin:32px 0'>
<a href='/' role='button' class='btn'>
Go to the web UI
</a>
</p>
</div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script src="https://unpkg.com/clipboard@2/dist/clipboard.min.js"></script>
<script>
new ClipboardJS('.copy-btn');
(
() => {
let btns = document.getElementsByClassName('copy-btn')
for (const button of btns) {
button.onclick = () => {
let old = button.innerHTML;
button.innerHTML = 'Copied!';
setTimeout(() => { button.innerHTML = old }, 500);
}
}
}
)()
</script>
<script src='/js-only.js'></script>
{% endblock %}

View File

@ -5,5 +5,5 @@ REDIS_PREFIX=bfile-
USERCONTENT_DIR=/opt/user_uploads USERCONTENT_DIR=/opt/user_uploads
CLEAN_DEL=30 minutes CLEAN_DEL=30min
CLEAN_ERRDEL=2 minutes CLEAN_ERRDEL=2min

View File

@ -10,7 +10,7 @@ RUN cargo b -r
RUN strip target/release/janitor && upx --best target/release/janitor RUN strip target/release/janitor && upx --best target/release/janitor
# --- deploy --- # --- deploy ---
FROM busybox:musl FROM alpine:3.17
RUN mkdir /config RUN mkdir /config
WORKDIR /config WORKDIR /config

46
resource/.air.toml Normal file
View File

@ -0,0 +1,46 @@
root = "."
testdata_dir = "testdata"
tmp_dir = "tmp"
[build]
args_bin = []
bin = "./tmp/main"
cmd = "go build -o ./tmp/main ."
delay = 1000
exclude_dir = ["assets", "tmp", "vendor", "testdata"]
exclude_file = []
exclude_regex = [ ".*\\.md" ]
exclude_unchanged = false
follow_symlink = false
full_bin = ""
include_dir = []
include_ext = ["go", "tpl", "tmpl", "html"]
include_file = []
kill_delay = "0s"
log = "build-errors.log"
poll = false
poll_interval = 0
post_cmd = []
pre_cmd = []
rerun = false
rerun_delay = 500
send_interrupt = false
stop_on_error = false
[color]
app = ""
build = "yellow"
main = "magenta"
runner = "green"
watcher = "cyan"
[log]
main_only = false
time = false
[misc]
clean_on_exit = false
[screen]
clear_on_rebuild = false
keep_scroll = true

5
resource/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
resourced.toml
resourced
.DS_Store
.env
tmp

22
resource/Dockerfile.prod Normal file
View File

@ -0,0 +1,22 @@
FROM golang:alpine3.17 as builder
WORKDIR /opt/build
COPY . .
RUN apk add --no-cache git musl-dev upx binutils
RUN go build . && \
strip resourced && \
upx resourced
FROM alpine:3.17
WORKDIR /opt/code
COPY --from=builder /opt/build/resourced /usr/bin/resourced
# Note
# -----
# Since this is running with prefork, don't
# forget to set --pid=host when running this app
CMD [ "sh", "-c", "/usr/bin/resourced" ]

15
resource/README.md Normal file
View File

@ -0,0 +1,15 @@
# resourceD
A simple microservice for serving resources
## Building
```sh
go build .
```
## Running
Either run it the normal way via the provided docker compose file, or `go run .`.
Also dont forget to create `resourced.toml`
## Configuration
The file `resourced.toml.example` serves as a both an example and reference, since it has a lot of comments.

View File

@ -0,0 +1,93 @@
openapi: 3.0.3
info:
title: ResourceD API
description: |-
This is the ResourceD API docs.
license:
name: GPLv3
url: https://www.gnu.org/licenses/gpl-3.0.en.html
version: "1.0"
servers:
- url: http://localhost/resource
tags:
- name: Data API
description: API for serving data
- name: System API
description: API for serving system data
- name: Internal API
description: API for microservices. Shouldn't be accessible via WAN
paths:
/{id}:
get:
description: Get a resource
summary: Get a resource. Send browser requests to this URL
tags:
- Data API
responses:
200:
description: Returns a resource in its binary data
302:
description: Resource is an external (http) link and the redirect is being forwarded to that link
404:
description: Not found
500:
description: Internal error
/info/is_enabled:
get:
description: Check if resourceD is enabled
summary: Check if resourceD is enabled
tags:
- System API
responses:
200:
description: Ok
content:
application/json:
schema:
type: boolean
example: true
/internal/check_resource:
post:
summary: Check if a resource(s) exist
description: |-
This method takes an array of resources and reports which ones exist.
See responses for more info.
tags:
- Internal API
requestBody:
description: |-
An array of strings, where each string is a resource ID
Let's assume that we want to check if two resources exist: `dev.blek.file.logo` and `org.nonexistant.resource`.
content:
application/json:
schema:
type: array
items:
type: string
example:
- dev.blek.file.logo
- org.nonexistant.resource
responses:
200:
description: |-
As in the request body example, let's assume that this method checked if these two resources exist:
- `dev.blek.file.logo`
- `org.nonexistant.resource`
As we can see, this returned only `dev.blek.file.logo` because its the only resource of the provided ones that exist.
content:
application/json:
schema:
type: object
additionalProperties:
type: object
properties:
type:
type: string
example:
dev.blek.file.logo:
type: image/png

20
resource/go.mod Normal file
View File

@ -0,0 +1,20 @@
module blek/resourced
go 1.21.3
require (
github.com/BurntSushi/toml v1.3.2 // indirect
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gofiber/fiber/v2 v2.50.0 // indirect
github.com/google/uuid v1.3.1 // indirect
github.com/klauspost/compress v1.16.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.50.0 // indirect
github.com/valyala/tcplisten v1.0.0 // indirect
golang.org/x/sys v0.13.0 // indirect
)

31
resource/go.sum Normal file
View File

@ -0,0 +1,31 @@
github.com/BurntSushi/toml v1.3.2 h1:o7IhLm0Msx3BaB+n3Ag7L8EVlByGnpq14C4YWiu/gL8=
github.com/BurntSushi/toml v1.3.2/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/andybalholm/brotli v1.0.5 h1:8uQZIdzKmjc/iuPu7O2ioW48L81FgatrcpfFmiq/cCs=
github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/gofiber/fiber/v2 v2.50.0 h1:ia0JaB+uw3GpNSCR5nvC5dsaxXjRU5OEu36aytx+zGw=
github.com/gofiber/fiber/v2 v2.50.0/go.mod h1:21eytvay9Is7S6z+OgPi7c7n4++tnClWmhpimVHMimw=
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.16.7 h1:2mk3MPGNzKyxErAw8YaohYh69+pa4sIQSC0fPGCFR9I=
github.com/klauspost/compress v1.16.7/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.50.0 h1:H7fweIlBm0rXLs2q0XbalvJ6r0CUPFWK3/bB4N13e9M=
github.com/valyala/fasthttp v1.50.0/go.mod h1:k2zXd82h/7UZc3VOdJ2WaUqt1uZ/XpXAfE9i+HBC3lA=
github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8=
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=

190
resource/main.go Normal file
View File

@ -0,0 +1,190 @@
package main
import (
"log"
"fmt"
"regexp"
"strings"
"net/http"
"io/ioutil"
"encoding/json"
)
import (
"github.com/BurntSushi/toml"
"github.com/gofiber/fiber/v2"
"github.com/dustin/go-humanize"
)
type Resource struct {
Url string `toml:"url"`
Proxied bool `toml:"proxied",default:false`
Mime string `toml:"mime"`
}
type CheckResourceType struct {
Type string `json:"type"`
}
func (self Resource) Get() ([]byte, error) {
return ioutil.ReadFile(self.Url[7:])
}
type ResourceDConfig struct {
Enabled bool `toml:"enabled"`
ListenURL string `toml:"listen_url"`
ProxyCacheMinSize string `toml:"proxy_cache_min_size",default:5MB`
}
type Config struct {
ResourceD ResourceDConfig `toml:"resourceD"`
Resource map[string]Resource `toml:"resource"`
}
func (self Config) Validate() int {
re, err := regexp.Compile(`^(file|http(s|))://`)
if err != nil { panic(err) }
for key, res := range self.Resource {
if ! re.MatchString(res.Url) {
panic(fmt.Sprintf("Resource %s has invalid URL: %s\nOnly file://, http:// and https:// URLs are allowed", key, res.Url))
}
}
return 0
}
func (self Resource) GetProxied() ([]byte, error) {
cached, exists := ProxyResourceCache[self.Url];
if exists {
return cached, nil
}
resp, err := http.Get(self.Url)
if err != nil { return make([]byte, 0, 0), err }
buf, err := ioutil.ReadAll(resp.Body)
if err != nil { return make([]byte, 0, 0), err }
// cache only those that are less than 5 mb
if len(buf) > ProxyCacheMinSize {
ProxyResourceCache[self.Url] = buf
}
return buf, nil
}
var ProxyResourceCache map[string][]byte = make(map[string][]byte)
var ProxyCacheMinSize int
func main() {
var conf Config
data, err := ioutil.ReadFile("resourced.toml")
if err != nil { panic(err) }
a, err := toml.Decode(string(data), &conf)
if err != nil { panic(err) }
_ = a
cache_min, err := humanize.ParseBytes(conf.ResourceD.ProxyCacheMinSize)
if err != nil { panic(err) }
ProxyCacheMinSize = int(cache_min)
conf.Validate()
if len(conf.Resource) == 0 {
fmt.Println("\x1b[33m[warn] No resources are specified\x1b[0m");
} else {
fmt.Println(fmt.Sprintf("[info] Loaded %d resources", len(conf.Resource)))
}
if ! conf.ResourceD.Enabled {
fmt.Println("\x1b[33m[warn] resourceD is disabled. No resources will be served\x1b[0m")
}
app := fiber.New(fiber.Config {
Prefork: true,
CaseSensitive: false,
StrictRouting: true,
ServerHeader: "",
AppName: "blek! File resourceD",
})
app.Get("/info/is_enabled", func (c *fiber.Ctx) error {
return c.JSON(conf.ResourceD.Enabled)
})
app.Use(func (c *fiber.Ctx) error {
if ! conf.ResourceD.Enabled {
return c.Status(fiber.StatusNotFound).SendString("ResourceD is disabled")
}
return c.Next()
})
app.Post("/internal/check_resource", func (c *fiber.Ctx) error {
resources := new([]string)
err := json.Unmarshal(c.Body(), &resources)
if err != nil {
log.Fatalln(err)
return c.Send([]byte("{ \"error\": \"500\"}"))
}
resources_exist := make(map[string]CheckResourceType)
for _, resource := range *resources {
if val, ok := conf.Resource[resource]; ok {
resources_exist[resource] = CheckResourceType {
Type: val.Mime,
}
}
}
marshaled, err := json.Marshal(resources_exist)
if err != nil {
log.Fatalln(err)
return c.Send([]byte("{ \"error\": \"500\"}"))
}
return c.Send([]byte(marshaled))
})
app.Get("/:id", func (c *fiber.Ctx) error {
res, exists := conf.Resource[c.Params("id")]
if ! exists {
return c.Status(fiber.StatusNotFound).SendString("Resource not found")
}
if ! strings.HasPrefix(res.Url, "file://") {
if res.Proxied {
data, err := res.GetProxied()
if err != nil {
log.Fatalln(err)
// we failed, send a redirect instead
// the next line would be the one with
// c.Location(res.Url)
} else {
c.Response().Header.SetContentType(res.Mime)
c.Response().Header.SetContentLength(len(data))
return c.Send(data)
}
}
c.Location(res.Url)
c.Status(302)
return nil
}
data, err := res.Get()
if err != nil {
panic(err)
}
c.Response().Header.SetContentType(res.Mime)
c.Response().Header.SetContentLength(len(data))
return c.Send(data)
})
log.Fatal(app.Listen(conf.ResourceD.ListenURL))
}

View File

@ -0,0 +1,47 @@
# The resourceD config
[resourceD]
# Whether to enable the resourceD.
# If this is false, resourceD will start but respond to
# all requests with 404
# It is false by default because resourceD is not required in a default installation.
enabled=true
# URL to listen on
listen_url="0.0.0.0:80"
# Minibal size of a file to be cached
# File size is parsed via this library:
# https://github.com/dustin/go-humanize
proxy_cache_min_size="5MB"
# Resource ID must be like a java package name
# At least one X.X. is required
#
# Examples:
# org.university.logo
# dev.indie_guy.logo
# com.pany.logo
# Test your names here: https://regex101.com/r/wQdOup/2
[resource."com.example.logo"]
# Can also be an external link
# If an external link is specified,
# the resource will be returned as a 302 redirect to the link
url="file:///some/where"
# File type, as according to this:
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types
mime="image/jpg"
# (if url is an http(s) url))
# Whether to proxy the resource.
# true - Send the resource like a regular file
# false - Send a 302 HTTP redirect to the URL (default)
#
# It is not recommended to set this to `true`
# unless you are referring to a resource that is
# available only in the local network
#
# proxied=true