The problem with making a custom web server is that you take responsibility for re-solving all the non-obvious security vulnerabilities. I always try to delegate as much network-facing code as possible to a mature implementation someone else wrote for that reason.

Here’s how I’d implement it, based on stuff I’ve done before:

1. Start with either Actix Web or Axum for the server itself.
2. Use std::thread to bring up mpv in a separate thread.
3. Use an async-capable channel implementation like flume as a bridge between the async and sync worlds.
4. If the async side needs to wait on the sync side, include the sending side of a tokio::sync::oneshot in the “job order” object your async code drops into the channel and then have the async task await the receiving side. That way, you can have the async task block on the some kind of completion signal from the sync thread without blocking the thread(s) underlying the task executor.

Wouldn’t you want your SSG to include a dev-server anyways? Zola has zola serve which even does incremental rebuilds, but something less sophisticated should be easy to add to your own (only took me a weekend to add to hinoki including rebuilds, though mostly starting the build from scratch on changes).

I don’t want the overhead of looping through an HTTP client and server implementation in places it doesn’t need to. I design my tooling based on a test target roughly comparable to the Raspberry Pi 4, performance-wise.

Have you investigated some of the options already now?

A bunch of other things came up, forcing me to put the project on the back burner.

(eg. Most recently (about a week ago), I had my 6-month-old boot drive go bad and it took me several days to rush-order a new NVMe drive, learn ZFSBootMenu, restore my backups, and redesign my backup strategy so that, when the original comes back from RMA, if the ZFS mirroring and snapshotting and the trick to mirror the EFI system partition isn’t enough to ensure high availability, a full, bootable backup of the NVMe pool’s contents can be restored in 2 hours or less with the sequential read performance of my first tier of backup being the bottleneck.)

missing flexibility for output paths has been an annoyance.

Hmm. We’ll see if I wind up using it. Avoiding deadlinks has been non-negotiable to the point where replicating my WordPress blog on a local httpd, spidering it, and logging the URLs I need to preserve has been one of the big hold-ups.

is that I found Zola to be quite hard to hack on

Hmm. Potentially a reason I’ll wind up making my own, given that I’ve written SSGs in Python before (eg. https://vffa.ficfan.org/ is on a homebrew Python SSG) and I’ve already got a single-page pulldown-cmark frontend I’ve gone way overboard on the features for and a basic task-specific Rust SSG for my mother’s art website that I can merge with it and generalize.

EDIT: Here’s a screenshot of what I mean by saying I’ve gone way overboard.

and Tera (its templating lang) to be a little buggy / much less elegant than minijinja API-wise.

Hmm. Noted. I think i’m using Tera for my mother’s SSG.

Re. link checking, have you seen lychee? When I found out about it, the priority of building my own link checker in my SSG (that was only an idea at that point, I think) basically dropped to zero :D

You accidentally re-used the link to the Zola issue tracker there. I have not yet checked out lychee and I’m getting a docs.rs error when clicking the examples link, so all I can say is that it’ll depend on how amenable it is to checking a site rooted in a file:// URL so I don’t need the overhead and complexity of spinning up an HTTP server to check for broken links.

Maybe a web of trust for audited dependencies would help. This version of this repo under this hash.

It’ll probably please you to know that the alternative lib.rs frontend for the crates repository integrates both cargo-vet and cargo-crev data via an “Audit” tab that appears on the pages for crates that have such information.

I’m using the web UI, so I’m assuming whatever broad-spectrum Markdown rendering library it uses has smart quote rendering turned on.

Edit: and just to be snarky: I didn’t type “…” I typed “…”. ;)

*chuckle* I think Lemmy typed those for you, because I typed three periods and got a Unicode ellipsis, and both of those are also unicode ellipses.

It still returns relative paths if the input was relative

False

and it doesn’t resolve “…”

I’ll assume you meant .., since ... is an ordinary filename. (Aside from the “who remembers …?” feature introduced in Windows 95’s COMMAND.COM where cd ... was shorthand for doing cd .. twice and you could omit the space after cd if your target was all dots.)

The reason it doesn’t do that is that, when symlinks get involved, /foo/bar/.. does not necessarily resolve to /foo and making that assumption could introduce a lurking security vulnerability in programs which use it.

Ahh, yeah. In the beginning, Rust was built around the idea that individual files and invoking rustc are internal details, only relevant for integration into some other build system like Bazel, and that “normal” Rust projects need to be inside a Cargo project structure.

There is in-development work to have official support for something along the lines of rust-script, but it’s still just that… in development. If you want to keep an eye on it, here is the tracking issue.

That’s not how it’s supposed to be.

but for example Vec::new() doesn’t highlight anything at all.

If I do Vec::new(foo, bar), I get expected 0 arguments, found 2 (rust-analyzer E0107).

but things like passing too many arguments into the println macro doesn’t throw an error.

I don’t get that either, but I’m still running with the Vim configuration I setup on my previous PC from 2011, where I turned off checks that require calling cargo check or cargo clippy in the background. From what I remember, a properly functioning default rust-analyzer config should pick up and display anything cargo check will catch and you can switch it to cargo clippy for even stricter results.

Or how shown in this example, it did not autocomplete the clone method, while also just accepting .clo; at the end of a String (it also didn’t autocomplete “String”).

I get clone(), clone_into(), and clone_from() as proposed completions for .clo on my as-you-type completions for foo where let foo = String::new(); and it proposed a whole bunch of things, with String at the top when I typed Stri. (eg. the stringify! macro, OsString, mixed in with various results from other crates in the project like serde)

Fair. That would have been more constructive… I think I didn’t do that because it still would have felt like encouraging off-topicness.

Signal to noise ratio.

Aside from possibly making them feel better, it doesn’t benefit anyone for them to drop into a topic about thing X and say nothing but “I use thing Y. I don’t like thing X.” and it wastes other people’s time either scrolling past it or clearing out their RSS reader, depending on how they follow things.

…and I could just as easily disparage those frameworks and give concrete reasons, but I don’t. If you don’t have something constructive to say, please be courteous and say nothing.

Depending on your preferences, there’s also Nom if you prefer parser combinators, or lalrpop or grmtools if you prefer LR(1) parsing.

Since reading Which Parsing Approach by Laurence Tratt (author of grmtools), my plan for my own parsing projects has been to use an LR(1) parser generator for the stronger compile-time guarantees.

Thanks. Being the biggest name, Zola is definitely on my list of things to investigate.

I’ll try to fit in sampling it at some point in the near future as a candidate for building on.

I just decided to finally double down and do the work to switch away from WordPress to GitHub Pages and:

• Jekyll is still hell to get running locally for testing without erroring out during the install
• Pelican seems like it’d be more trouble than it’s worth to get what I want
• I insist that no links be broken in the switchover (Doing this to my standards was a big part of what I wound up procrastinating, since I basically need to install WordPress locally and then write something which spiders the entire site and verifies that each path is also present in the new site and the page’s contents are identical when run through a filter to cut away the site template and normalize any irrelevant rendering differences.)
• I already have a pulldown-cmark-based CLI that I wrote a couple of years ago to render single documents and it’d be nice to retrofit it (or at least its features) onto something Rust-based for my blog. (Hell, just a couple of days ago, after implementing support for shortcodes, I got carried away implementing a complete set of shortcodes for rendering depictions of gamepad buttons like :btn-l-snes: within passages of text. Bit of a shame, though, that I’d have to either patch pulldown-cmark or maintain the smart punctuation and strikethrough extensions externally, if I want to hook in shortcodes early enough in the pipeline to be able to implement Compose key-inspired ones like :'e:/:e': → é or :~n:/:n~: → ñ without breaking things.)
• Since my plans for comments are, to the best of my knowledge, unique, I need something in a language I’m willing to hack on and potentially maintain my own fork of. (Jekyll would have been achieved via a preprocessor.)
• I want something where I’m at least willing to port the internal broken link detection from one of my old bespoke Python static site generators, which means either Python or Rust. (Ideally, I’ll also re-create the support for performing HTML and CSS validation on the generated output.)
• Given how many things I either have in my existing single-page renderer (eg. automatic ToC generation with a bespoke scrollspy implementation, Syntect integration, ```svgbob fenced code blocks which produce rendered diagrams, <price></price> tags which provide currency-conversion estimation tooltips with the exchange rate defined in a central location, etc.) or have plans for (eg. plotters-generated charts with some kind contributed extension equivalent to matplotlib’s xkcd mode because it’s important, Wikipedia-style infobox sidebars, etc.), I want to experiment with a WebAssembly-based plugin API so I’m not throwing the kitchen sink in.

Lemmy hangs whenever I try to post my response (I suspect it doesn’t like the length), so here’s a link to it on Github Gist:

https://gist.github.com/ssokolow/16c9311573eabc7343ff7ff2cc3513b3

It begins as follows and I’ve tried to hyperlink my sources as often as possible:

I’ll try to fill in some of the knowledge gaps and respond to some of your answers from a more user-centric perspective.

He recently did one about how he and his team set up a fake bitcoin site that would direct scammers to a fake support hotline when they try to withdraw the fake bitcoin, where they’d get stuck running in circles in a voice mail menu maze chasing the illusory bitcoin payout.

I Trapped 200 Scammers in an Impossible Maze

As one commenter put it, “I love how Kit has evolved over the years to find out the best way of making scammers go crazy is to treat them basically the same way Comcast treats their customers.”

Moxie Marlinspike’s My first impressions of web3 is also a very relevant thing to share.

As a sampler of the points made, web3 is already re-centralizing around gatekeepers because the average person doesn’t want to run their own server (or, in the blockchain case, host their own full copy of the blockchain) and, if the supermajority of users can’t see you because the gatekeepers block you, then it doesn’t really matter that you’re technically still up.

The takeaway on that particular point is that pushing for more and easier data portability is probably the best route in the face of how real-world users behave. (eg. anything stored in a git repository, including GitHub project wiki contents, is a great example of that. You’ve got your data locally with a simple git clone and you can upload it to a competing service with a simple git push.)