From 3b5dcc3c56e2818b93f55ec40d6c35d605d54a49 Mon Sep 17 00:00:00 2001 From: luxagraf Date: Wed, 24 Feb 2021 20:58:20 -0500 Subject: rearranging, moving the single large file workflow. --- src/guide-to-switching-i3-to-sway.txt | 132 --------------------- src/indie-web-co.txt | 36 ++++++ src/ranger.txt | 63 ---------- src/solving-common-nextcloud-problems.txt | 92 -------------- src/src-guide-to-switching-i3-to-sway.txt | 132 +++++++++++++++++++++ src/src-ranger.txt | 63 ++++++++++ src/src-solving-common-nextcloud-problems.txt | 92 ++++++++++++++ ...rc-why-i-built-my-own-mailing-list-software.txt | 52 ++++++++ src/why-i-built-my-own-mailing-list-software.txt | 52 -------- 9 files changed, 375 insertions(+), 339 deletions(-) delete mode 100644 src/guide-to-switching-i3-to-sway.txt create mode 100644 src/indie-web-co.txt delete mode 100644 src/ranger.txt delete mode 100644 src/solving-common-nextcloud-problems.txt create mode 100644 src/src-guide-to-switching-i3-to-sway.txt create mode 100644 src/src-ranger.txt create mode 100644 src/src-solving-common-nextcloud-problems.txt create mode 100644 src/src-why-i-built-my-own-mailing-list-software.txt delete mode 100644 src/why-i-built-my-own-mailing-list-software.txt (limited to 'src') diff --git a/src/guide-to-switching-i3-to-sway.txt b/src/guide-to-switching-i3-to-sway.txt deleted file mode 100644 index 33e7742..0000000 --- a/src/guide-to-switching-i3-to-sway.txt +++ /dev/null @@ -1,132 +0,0 @@ -I recently made the switch from the [i3 tiling window manager](https://i3wm.org/) to [Sway](https://swaywm.org/), a Wayland-based i3 clone. I still [run Arch Linux on my personal machine](/src/why-i-switched-arch-linux), so all of this is within the context of Arch. - -I made the switch for a variety of reasons. There's the practical: Sway/Wayland gives me much better battery life on my laptop. As well as the more philosophical: Sway's lead developer Drew Devault's take on code is similar to mine[^1] (e.g. [avoid traumatic changes](https://drewdevault.com/2019/11/26/Avoid-traumatic-changes.html) or [avoid dependencies](https://drewdevault.com//2020/02/06/Dependencies-and-maintainers.html)), and after reading his blog for a year he's someone whose software I trust. - -I know some people would think this reason ridiculous, but it's important to me that the software I rely on be made by people I like and trust. Software is made by humans, for humans. The humans are important. And yes, it goes the other way too. I'm not going to name names, but there are some theoretically good software out there that I refuse to use because I do not like or trust the people who make it. - -When I find great software made by people who seem trustworthy, I use it. So I switched to Sway and it's been a good experience. - -Sway and Wayland have been very stable in my use. I get about 20 percent more out of my laptop battery. That seems insane to me, but as someone who [lives almost entirely off solar power](/1969-dodge-travco-motorhome) it's a huge win I can't ignore. - -### Before You Begin - -I did not blindly switch to Sway. Or rather I did and that did not go well. I switched back after a few hours and started doing some serious searching, both the search engine variety and the broader, what am I really trying to do here, variety. - -The latter led me to change a few tools, replace some things, and try some new work flows. Not all of it was good. I could never get imv to do the things I can with feh for instance, but mostly it was good. - -One thing I really wanted to do was avoid XWayland (which allows apps that need X11 to run under Wayland). Wherever I could I've opted for applications that run natively under Wayland. There's nothing wrong with XWayland, that was just a personal goal, for fun. - -Here's my notes on making the transition to Wayland along with the applications I use most frequently. - -##### Terminal - -I do almost everything in the terminal. I write in Vim, email with mutt, read RSS feeds with newsboat, listen to music with mpd, and browse files with ranger. - -I tested quite a few Wayland-native terminals and I really like [Alacritty](https://github.com/alacritty/alacritty). Highly recommended. [Kitty](https://github.com/kovidgoyal/kitty) is another option to consider. - -That said, I am sticking with urxvt for now. There are two problems for me with Alacritty. First off Vim doesn't play well with the Wayland clipboard in Alacritty. Second, Ranger will not show image previews in Alacritty. Both things work as I would expect in urxvt. I'd like to find a solution to these issues because I like Alacritty better than urxvt, but so far nothing has made Vim work and that one is a deal breaker for me. - -##### Launcher - -I've always used dmenu to launch apps and grab passwords from pass. It's simple and fast. Unfortunately dmenu is probably never going to run natively in Wayland. - -I tested rofi, wofi, and other potential replacements, but I did not like any of them. Somewhere in my search for a replacement launcher I ran across [this post](https://medium.com/njiuko/using-fzf-instead-of-dmenu-2780d184753f) which suggested just calling up a small terminal window and piping a list of applications to [fzf](https://github.com/junegunn/fzf), a blazing fast search tool. - -That's what I've done and it works great. I created a keybinding to launch a new instance of Alacritty with a class name that I use to resize the window. Then within that small Alacritty window I call `compgen` to get a list of executables, then sort it to eliminate duplicates, and pass the results to fzf. Here's the code in my Sway config file: - -~~~console -bindsym $mod+Space exec alacritty --class 'launcher' --command bash -c 'compgen -c | sort -u | fzf | xargs -r swaymsg -t command exec' - -for_window [app_id="^launcher$"] floating enable, border none, resize set width 25 ppt height 20 ppt, move position 0 px 0 px -~~~ - -These lines together will open a small terminal window in the upper left corner of the screen with a fzf search interface. I type, for example, "dar" and Darktable comes up. I hit return, the terminal window closes, and Darktable launches. It's as simple as dmenu and requires no extra applications (since I was already using fzf in Vim). - -##### Browsers - -I mainly use [qutebrower](https://qutebrowser.org/), supplemented by [Vivaldi](https://vivaldi.com/)[^2] for research because having split screen tabs is brilliant for research. I also use [Firefox Developer Edition](https://www.mozilla.org/en-US/firefox/developer/) for any web development work, because the Firefox dev tools are far superior to anything else. - -Qutebrowser and Firefox work great under Wayland. In the case of qutebrowser though you'll need to set a few shell variables to get it to start under Wayland, out of the box it launches with XWayland for some reason. Here's what I added to `.bashrc` to get it to work: - -~~~bash -export XDG_SESSION_TYPE=wayland -export GDK_BACKEND=wayland -~~~ - -One thing to bear in mind if you do have a lot of X11 apps still is that with this in your shell you'll need to reset the `GDK_BACKEND` to X11 or those apps won't launch. Instead you'll get an error, `Gtk-WARNING **: cannot open display: :0`. To fix that error you'll need to reset `GDK_BACKEND=x11`, then launch your X11 app. - -There are several ways you can do this, but I prefer to override apps in `~/bin` (which is on my $PATH). So, for example, I have a file named `vivaldi` in `~/bin` that looks like this: - -~~~bash -#! /bin/sh -GDK_BACKEND=x11 vivaldi-stable -~~~ - -Alternately you could just create a bash alias like `alias vivaldi-stable="GDK_BACKEND=x11 vivaldi-stable"`, but I prefer my `~/bin` method since I override a bunch of other apps too for other reasons and this way everything is in one spot. - -Note that for me this is easier, because the only apps I'm using that need X11 are Vivaldi and Slack. If you have a lot of X11 apps, you're probably better off making qutebrowser the special case by launching it like this: - -~~~bash -GDK_BACKEND=wayland qutebrowser -~~~ - -##### Clipboard - -I can't work without a clipboard manager, I keep the last 200 things I've copied, and I like to have things permanently stored as well. - -Clipman does a good job of saving clipboard history. - -You need to have wl-clipboard installed since Clipman reads and writes to and from that. I also use wofi instead of the default dmenu for viewing and searching clipboard history. Here's how I set up clipman in my Sway config file: - -~~~bash -exec wl-paste -t text --watch clipman store --max-items=60 --histpath="~/.local/share/clipman.json" -bindsym $mod+h exec clipman pick --tool="wofi" --max-items=30 --histpath="~/.local/share/clipman.json" -~~~ - -Clipman does not, however, have a way to permanently store bits of text. That's fine. Permanently stored bits of frequently used text are really not all that closely related to clipboard items and lumping them together in a single tool isn't a very Unix-y approach. Do one thing, do it well. - -For snippets I ended up bending [pet](https://github.com/knqyf263/pet), the "command line snippet manager" a little and combining it with the small launcher-style window idea above. So I store snippets in pet, mostly just `printf "my string of text"`, call up an Alacritty window, search, and hit return to inject the pet snippet into the clipboard. Then I paste it were I need it. - -##### Volume Controls - -Sway handles volume controls with pactl. Drop this in your Sway config file and you should be good: - -~~~bash -bindsym XF86AudioRaiseVolume exec pactl set-sink-volume @DEFAULT_SINK@ +5% -bindsym XF86AudioLowerVolume exec pactl set-sink-volume @DEFAULT_SINK@ -5% -bindsym XF86AudioMute exec pactl set-sink-mute @DEFAULT_SINK@ toggle -bindsym XF86AudioMicMute exec pactl set-source-mute @DEFAULT_SOURCE@ toggle -~~~ - -##### Brightness - -I like [light](https://github.com/haikarainen/light) for brightness. Once it's installed these lines from my Sway config file assign it to my brightness keys: - -~~~bash -bindsym --locked XF86MonBrightnessUp exec --no-startup-id light -A 10 -bindsym --locked XF86MonBrightnessDown exec --no-startup-id light -U 10 -~~~ - -### Quirks, Annoyances And Things I Haven't Fixed - -There have been surprisingly few of these, save the Vim and Ranger issues mentioned above. - -I haven't found a working replacement for xcape. The only thing I used xcape for was to make my Cap Lock key dual-function: press generates Esc, hold generates Control. So far I have not found a way to do this in Wayland. There is ostensibly [caps2esc](https://gitlab.com/interception/linux/plugins/caps2esc), but it's poorly documented and all I've been able to reliably do with it is crash Wayland. - -I miss xcape. I've reverted to `inoremap jj ` for vim, but if you have a solution, please do let me know in the comments. - -I've been using Waybar as a status bar and it works well enough. To be honest I haven't put in the time to get it exactly how I want it, but it's close enough. My one gripe is that Slack doesn't show up in the icon tray, but I hate Slack and only use it because of work, so my motivation to debug that issue is very low. - -Surprisingly, those are the only problems I can think of right now. I've been using Sway exclusively for almost two months now and I have no reason or desire to ever go back. - -### Helpful pages: - -- **[Sway Wiki](https://github.com/swaywm/sway/wiki)**: A good overview of Sway, config examples (how to replicate things from i3), and application replacement tips for i3 users (like this fork of [redshift](https://github.com/minus7/redshift/tree/wayland) with support for Wayland). -- **[Arch Wiki Sway Page](https://wiki.archlinux.org/index.php/Sway)**: Another good Sway resource with solutions to a lot of common stuff: set wallpaper, take screenshots, HiDPI, etc. -- **[Sway Reddit](https://old.reddit.com/r/swaywm/)**: There's some useful info here, worth searching if you run into issues. Also quite a few good tips and tricks from fellow Sway users with more experience. -- **[Drew Devault's Blog](https://drewdevault.com/)**: He doesn't always write about Sway, but he does give updates on what he's working on, which sometimes has details on Sway updates. - - -[^1]: That's not to imply there's anything wrong with the i3 developers. - -[^2]: Vivaldi would be another good example of me trusting a developer. I've been interviewing Jon von Tetzchner for many years, all the way back to when he was at Opera. I don't always see eye to eye with him (I wish Vivaldi were open source) but I trust him, so I use Vivaldi. It's the only software I use that's not open source (not including work, which requires quite a few closed source crap apps). diff --git a/src/indie-web-co.txt b/src/indie-web-co.txt new file mode 100644 index 0000000..9924aae --- /dev/null +++ b/src/indie-web-co.txt @@ -0,0 +1,36 @@ +Here's a disturbing factoid: **the world’s ten richest men have made $540 billion so far during the pandemic.** Amazon founder Jeff Bezos' worth went up so much between March and September 2020 that he could afford to give all 876,000 Amazon employees a $105k bonus and still have as much money as he had before the pandemic started ([source](https://oxfamilibrary.openrepository.com/bitstream/handle/10546/621149/bp-the-inequality-virus-summ-250121-en.pdf)). + +What does that have to do with code? Well, some of my code used to run on Amazon services. Some of my money is in Jeff Bezos' pocket. I was contributing to the economic inequality that Amazon enables. I decided I did not want to do that. + +But more than I didn't want to contribute to Amazon's bottom line, I *wanted* to contribute to someone's bottom line, the emphasis being on *someone*. I wanted to redirect the money I was already spending to small businesses, businesses that need the revenue. + +We can help each other instead of Silicon Valley billionaires. + +Late last year at [work](https://www.wired.com/author/scott-gilbertson/) we started showcasing some smaller, local businesses in affiliate links. It was a pretty simple idea, find some small companies in our communities making worthwhile things and support them by telling others. + +One woman whose company I linked to called it "life-changing." It's so strange to me that an act as simple as pasting some HTML into the right text box can changed someone's life. That's amazing. I bring this up not to toot my own horn, but to say that every day there are ways in which you can use the money you spend to help real people trying to make a living. If you've ever charged a little for a web service you probably know how much of a big deal even one more customer means. I want to be that one more customer for someone. + +My online expenses aren't much, just email, web hosting, storage space, and domain registration. I wanted to find some small business replacements for the megacorps I was using. + +I did a ton of research. Web hosting and email servers are tricky, these are critical things that run my business and my wife's business. It's great to support small businesses, but above all the services have to *work*. Luckily for us the forums over at [Low End Talk](https://www.lowendtalk.com/) are full of ideas and long term reviews of exactly these sorts of business -- small companies offering cheap web hosting, email hosting, and domain registration. + +After a few late nights digging through threads, finding the highlights, and then more research elsewhere on the web, I settled on [BuyVM](https://buyvm.net/) for my web hosting. The owner Francisco is very active on Low End Talk and, in my experience for the last three months, is providing a great service *for less* than I was paying at Vultr. It was so much less I was able to get a much larger block storage disk and have more room for my backups, which eliminated my need for Amazon S3/Glacier as well[^2]. I highly recommend BuyVM for your VPS needs. + +For email hosting I actually was already using a small company, [Migadu](https://www.migadu.com/). I liked their service, and I still recommend them if the pricing works for you, but they discountinued the plan I was on and I would have had to move to a more expensive plan to retain the same functionality. + +I jumped ship from Migadu during Black Friday because another small email provider I had heard good things about was having a deal: $100 for life. At that price, so long as it stays in business for 2 years, I won't loss any money. I moved my email to [MxRoute](https://mxroute.com/) and it has been excellant. I liked it so much I bought my parents a domain and freed them from Google. Highly recommend MxRoute. + +That left just one element of my web stack at Amazon: domain registration. I'll confess I gave up here. Domain registration are not a space filled with small companies (which to me is like 2-8 people). I gave up. And complained to a friend, who said, try harder. So I did and discovered [Porkbun](https://porkbun.com/), the best domain registrar I've used in the past two decades. I moved my small collection of domain over at the beginning of the year and it was a seamless, super-smooth transition. It lives up to its slogan: "an oddly safisfying experience." + +And those are my recommendations for small businesses you can support *and* still have a great technology stack: [Porkbun](https://porkbun.com/) (domain registration), [MxRoute](https://mxroute.com/) (email hosting), and [BuyVM](https://buyvm.net/) (VPS hosting). + +The thing I didn't replace was AWS CloudFront. I don't have enough traffic to warrant a CDN, so I just dropped it. If I ever change my mind about that, based on my research, I'll go with [KeyCDN](https://www.keycdn.com/pricing), or possible [Hostry](https://hostry.com/products/cdn/). + +I also haven't found a reliable replacement for SES, which I use to send my newsletters. I wish Sendgrid would spin off a company for non-transational email, but I don't see that happening. I could write another 5,000 words on how the big email providers totally, purposefully fucked up the best distributed communication system around. But I will spare you. + +The point is, these are three small companies providing useful services we developers need. If you're feeling like you'd rather your money went to people trying to make cool, useful stuff, rather than massive corporations, give them a try. If you have other suggestions drop them in the comments and maybe I can put together some sort of larger list. + +[Note: none of these links are affiliate links, just services I actually use and therefore recommend.] + +[^1]: This is something I'd like to do more, unfortunately there are not cottage industries for most of the things I write about (cameras, laptops, etc). Still, you do what you can I guess. +[^2]: I have a second cloud-based backup stored in Backblaze's B2 system. Backblaze is not a small company by any means, but it's one that, from the research I've been able to do, seems ethically run and about as decent as a corporation can be these days. diff --git a/src/ranger.txt b/src/ranger.txt deleted file mode 100644 index f66a707..0000000 --- a/src/ranger.txt +++ /dev/null @@ -1,63 +0,0 @@ -[Ranger](http://nongnu.org/ranger/) is a terminal-based file browser with Vim-style keybindings. It uses ncurses and can hook into all sorts of other command line apps to create an incredibly powerful file manager. - -If you prefer a graphical experience, more power to you. I'm lazy. Since I'm already using the terminal for 90 percent of what I do, it make sense to not leave it just because I want to browse files. - -The keyword here for me is "browse." I do lots of things to files without using Ranger. Moving, copying, creating, things like that I tend to do directly with `cp`, `mv`, `touch`, `mkdir` and so on. But sometimes you want *browse* files, and in those cases Ranger is the best option I've used. - -That said, Ranger is something of a labyrinth of commands and keeping track of them all can be overwhelming. If I had a dollar for every time I've searched "show hidden files in Ranger" I could buy you a couple beers (the answer, fellow searchers, is `zh`). - -I'm going to assume you're familiar with the basics of movement in Ranger like `h`, `j`, `k`, `l`, `gg`, and `G`. Likewise that you're comfortable with `yy`, `dd`, `pp`, and other copy, cut, and paste commands. If you're not, if you're brand new to ranger, check out [the official documentation](https://github.com/ranger/ranger/wiki/Official-user-guide) which has a pretty good overview of how to do all the basic stuff you'll want to do with a file browser. - -Here's a few less obvious shortcuts I use all the time. Despite some overlap with Vim, I do not find these particularly intuitive, and had a difficult time remembering them at first: - -- `zh`: toggle hidden files -- `gh`: go home (`cd ~/`) -- `oc`: order by create date (newest at top) -- `7j`: jump down seven lines (any number followed by j or k will jump that many lines) -- `7G`: jump to line 7 (any number followed by `G` will jump to that line -- `.d`: show only directories -- `.f`: show only files -- `.c`: clear any filters (such as either of the previous two commands) - -Those are handy, but if you really want to speed up Ranger and bend it to the way you work, the config file is your friend. What follows are a few things I've done to tweak Ranger's config file to make my life easier. - -###Ranger Power User Recommendations - -Enabling line numbers was a revelation for me. Open `~/.config/ranger/rc.conf` and search for `set line_numbers` and change the value to either `absolute` or `relative`. The first numbers from the top no matter what, the `relative` option sets numbers relative to the cursor. I can't stand relative, but absolute works great for me, YMMV. - -Another big leap forward in my Ranger productivity came when I discovered local folder sorting options. As noted above, typing `oc` changes the sort order within a folder to sort by date created[^1]. While typing `oc` is pretty easy, there are some folders that I *always* want sorted by date modified. That's easily done with Ranger's `setlocal` config option. - -Here's a couple lines from my `rc.conf` file as an example: - -~~~bash -setlocal path=~/notes sort mtime -setlocal path=~/notes/reading sort mtime -~~~ - -This means that every time I open `~/notes` or `~/notes/reading` the files I've worked with most recently are at the top (and note that you can also use `sort_reverse` instead of `sort`). That puts the most recently edited files right at the top where I can find them. - -Having my most recent notes at the top of the pane is great, but what makes it even more useful is having line wrapped file previews so I don't even need to open the file to read it. To get that I currently use the latest Git version of Ranger which I installed via [Arch Linux's AUR](https://aur.archlinux.org/packages/ranger-git/). - -This feature, which is invaluable to me since one of my common use cases for Ranger is quickly scanning a bunch of text files, has been [merged to master](https://github.com/ranger/ranger/pull/1322), but not released yet. If you don't [use Arch Linux](/src/why-i-switched-arch-linux) you can always build from source, or you can wait for the next release which should include an option to line wrap your previews. - -###Bookmarks - -Part of what makes Ranger incredibly fast are bookmarks. With two keystrokes I can jump between folders, move/copy files, and so on. - -To set a bookmark, navigate to the directory, then hit `m` and whatever letter you want to serve as the bookmark. Once you've bookmarked it, type ``` to jump straight to that directory. I try to use Vim-like mnemonics for my bookmarks, e.g. ``d` takes me to documents, ``n` takes me to `~/notes`, ``l` takes me to the dev folder for this site, and so on. As with the other commands, typing just ``` will bring up a list of your bookmarks. - -###Conclusion - -Ranger is incredibly powerful and almost infinitely customizable. In fact I don't think I really appreciated how customizable it was until I wrote this and dug a little deeper into all the ways you can map shell scripts to one or two character shortcuts. It can end up being a lot to keep track of though. I suggest learning maybe one or two new shortcuts a week. When you know longer have to think abut them, move on to the next couple. - -Or you can do what I do, wait until you have something you want to do, but don't know how, figure out how to do it, then write it down so you remember it. - -####Shoulders Stood Upon - -* [Dquinton's Ranger setup details](http://dquinton.github.io/debian-install/config/ranger.html) - I have no idea who this person is, but their Ranger setup and detailed notes was hugely helpful. -* [Ranger documentation](https://ranger.github.io/ranger.1.html) - The docs have a pretty good overview of the options available, though sometimes it's challenging to translate that into real world use cases. -* [Arch Wiki Ranger page](https://wiki.archlinux.org/index.php/Ranger) - Where would we be without the Arch Wiki? - - - -[^1]: In fact, just type `o` and you'll get a list of other sorting options (and if you know what `normal` means, drop me a comment below, I'm still trying to figure out what that means). diff --git a/src/solving-common-nextcloud-problems.txt b/src/solving-common-nextcloud-problems.txt deleted file mode 100644 index b32a629..0000000 --- a/src/solving-common-nextcloud-problems.txt +++ /dev/null @@ -1,92 +0,0 @@ -I love [NextCloud](https://nextcloud.com). Nextcloud allows me to have all the convenience of Dropbox, but hosted by me, controlled by me, and customized to suit my needs. I mainly use the file syncing, calendar, and contacts features, but Nextcloud can do a crazy amount of things. - -The problem with NextCloud, and maybe you could argue that this is the price you pay for the freedom and control, is that I find it requires a bit of maintenance to keep it running smoothly. Nextcloud does some decidedly odd things from time to time, and knowing how to deal with them can save you some disk space and maybe avoid syncing headaches. - -I should note, that while I call these problems, I **have never lost data** using Nextcloud. These are really more annoyances and some ways to prevent them than *problems*. - -### How to Get Rid of Huge Thumbnails in Nextcloud - -If Nextcloud is taking up more disk space than you think it should, or your Nextcloud storage space is just running low, the first thing to check is the image thumbnails directory. - -At one point I poked around in the Nextcloud `data` directory and found 11-gigabytes worth of image previews for only 6-gigabytes worth of actual images stored. That is crazy. That should never happen. - -Nextcloud's image thumbnail defaults err on the side of "make it look good in the browser" where as I prefer to err on the side of keep it really small. - -I did some research and came up with a few solutions. First, it looks like my runaway 11-gigabyte problem might have been due to a bug in older versions of Nextcloud. Ideally I will not hit that issue again. But, I don't admin servers with hope and optimism, so I figured out how to tell Nextcloud to generate smaller image previews. I almost never look at the images within the web UI, so I really don't care about the previews at all. I made them much, much smaller than the defaults. Here's the values I use: - -~~~bash -occ config:app:set previewgenerator squareSizes --value="32 256" -occ config:app:set previewgenerator widthSizes --value="256 384" -occ config:app:set previewgenerator heightSizes --value="256" -occ config:system:set preview_max_x --value 500 -occ config:system:set preview_max_y --value 500 -occ config:system:set jpeg_quality --value 60 -occ config:app:set preview jpeg_quality --value="60" -~~~ - -Just ssh into your Nextcloud server and run all these commands. If you followed the basic Nextcloud install instructions you'll want to run these as your web server user. For me, with NextCloud running on Debian 10, the full command looks like this: - -~~~bash -sudo -u www-data php /var/www/nextcloud/occ config:app:set previewgenerator squareSizes --value="32 256" -sudo -u www-data php /var/www/nextcloud/occ config:app:set previewgenerator widthSizes --value="256 384" -# and so on, running all the commands listed above -~~~ - -This assumes you installed Nextcloud into the directory `/var/www/nextcloud`, if you installed it somewhere else, adjust the path to the Nextcloud command line tool `occ`. - -That will stop Nextcloud from generating huge preview files. So far so good. I deleted the existing previews and reclaimed 11-gigabytes. Sweet. You can pre-generate previews, which will make the web UI faster if you browse images in it. I do not, so I didn't generate any previews ahead of time. - -### How to Solve `File is Locked` Issues in Nextcloud - -No matter what I do, I always end up with locked file syncing issues. Researching this led me to try using Redis to cache things, but that didn't help. I don't know why this happens. I blame PHP. When in doubt, blame PHP. - -Thankfully it doesn't happen very often, but every six months or so I'll see an error, then two, then they start piling up. Here's how to fix it. - -First, put Nextcloud in maintenance mode (again, assuming Debian 10, with Nextcloud in the `/var/www/nextcloud` directory): - -~~~bash -sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on -~~~ - -Now we're going directly into the database. For me that's Postgresql. If you use Mysql or Mariadb, you may need to adjust the syntax a little. - -~~~bash -psql -U yournextclouddbuser -hlocalhost -d yournextclouddbname -password: -nextclouddbname=> DELETE FROM oc_file_locks WHERE True; -~~~ - -That should get rid of all the locked file problems. For a while anyway. - -Don't forget to turn maintenance mode off: - -~~~bash -sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off -~~~ - -### Force a File Re-Scan - -If you frequently add and remove folders from Nextcloud, you may sometimes run into issues. I usually add a folder at the start of a new project, and then delete it when the project is finished. Mostly this just works, even with shared folders, on the rare occasion that I used them, but sometimes Nextcloud won't delete a folder. I have no idea why. It just throws an unhelpful error in the web admin and refuses to delete the folder from the server. - -I end up manually deleting it on the server using: `rm -rf path/to/storage/folder`. Nextcloud however, doesn't always seem to notice that the folder is gone, and still shows it in the web and sync client interfaces. The solution is to force Nextcloud to rescan all its files with this command: - -~~~bash -sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on -sudo -u www-data php /var/www/nextcloud/occ files:scan --path="yournextcloudusername/files/NameOfYourExternalStorage" -sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off -~~~ - -Beware that on large data directories this can take some time. It takes about 30 seconds to scan my roughly 30GB of files. - -### Mostly Though, Nextcloud is Awesome - -Those are three annoyances I've hit with Nextcloud over the years and the little tricks I've used to solve them. Lest anyone think I am complaining, I am not. Not really anyway. The image thumbnail thing is pretty egregious for a piece of software that aims to be enterprise grade, but mostly Nextcloud is pretty awesome. - -I rely on Nextcloud for files syncing, Calendar and Contact hosting, and keeping my notes synced across devices. Aside from these three things, I have never had a problem. - -####Shoulder's Stood Upon - -* [Nextcloud's documentation](https://docs.nextcloud.com) isn't the best, but can help get you pointed in the right direction. -* I tried a few different solutions to the thumbnail problem, especially helpful was this post on [Understanding and Improving Nextcloud Previews](https://ownyourbits.com/2019/06/29/understanding-and-improving-nextcloud-previews/), but nachoparker. -* The [file lock solution](https://help.nextcloud.com/t/file-is-locked-how-to-unlock/1883) comes from the Nextcloud forums. -* The solution to scanning external storages comes from the [Nextcloud forums](https://help.nextcloud.com/t/automate-occ-filescan/35282/4). diff --git a/src/src-guide-to-switching-i3-to-sway.txt b/src/src-guide-to-switching-i3-to-sway.txt new file mode 100644 index 0000000..33e7742 --- /dev/null +++ b/src/src-guide-to-switching-i3-to-sway.txt @@ -0,0 +1,132 @@ +I recently made the switch from the [i3 tiling window manager](https://i3wm.org/) to [Sway](https://swaywm.org/), a Wayland-based i3 clone. I still [run Arch Linux on my personal machine](/src/why-i-switched-arch-linux), so all of this is within the context of Arch. + +I made the switch for a variety of reasons. There's the practical: Sway/Wayland gives me much better battery life on my laptop. As well as the more philosophical: Sway's lead developer Drew Devault's take on code is similar to mine[^1] (e.g. [avoid traumatic changes](https://drewdevault.com/2019/11/26/Avoid-traumatic-changes.html) or [avoid dependencies](https://drewdevault.com//2020/02/06/Dependencies-and-maintainers.html)), and after reading his blog for a year he's someone whose software I trust. + +I know some people would think this reason ridiculous, but it's important to me that the software I rely on be made by people I like and trust. Software is made by humans, for humans. The humans are important. And yes, it goes the other way too. I'm not going to name names, but there are some theoretically good software out there that I refuse to use because I do not like or trust the people who make it. + +When I find great software made by people who seem trustworthy, I use it. So I switched to Sway and it's been a good experience. + +Sway and Wayland have been very stable in my use. I get about 20 percent more out of my laptop battery. That seems insane to me, but as someone who [lives almost entirely off solar power](/1969-dodge-travco-motorhome) it's a huge win I can't ignore. + +### Before You Begin + +I did not blindly switch to Sway. Or rather I did and that did not go well. I switched back after a few hours and started doing some serious searching, both the search engine variety and the broader, what am I really trying to do here, variety. + +The latter led me to change a few tools, replace some things, and try some new work flows. Not all of it was good. I could never get imv to do the things I can with feh for instance, but mostly it was good. + +One thing I really wanted to do was avoid XWayland (which allows apps that need X11 to run under Wayland). Wherever I could I've opted for applications that run natively under Wayland. There's nothing wrong with XWayland, that was just a personal goal, for fun. + +Here's my notes on making the transition to Wayland along with the applications I use most frequently. + +##### Terminal + +I do almost everything in the terminal. I write in Vim, email with mutt, read RSS feeds with newsboat, listen to music with mpd, and browse files with ranger. + +I tested quite a few Wayland-native terminals and I really like [Alacritty](https://github.com/alacritty/alacritty). Highly recommended. [Kitty](https://github.com/kovidgoyal/kitty) is another option to consider. + +That said, I am sticking with urxvt for now. There are two problems for me with Alacritty. First off Vim doesn't play well with the Wayland clipboard in Alacritty. Second, Ranger will not show image previews in Alacritty. Both things work as I would expect in urxvt. I'd like to find a solution to these issues because I like Alacritty better than urxvt, but so far nothing has made Vim work and that one is a deal breaker for me. + +##### Launcher + +I've always used dmenu to launch apps and grab passwords from pass. It's simple and fast. Unfortunately dmenu is probably never going to run natively in Wayland. + +I tested rofi, wofi, and other potential replacements, but I did not like any of them. Somewhere in my search for a replacement launcher I ran across [this post](https://medium.com/njiuko/using-fzf-instead-of-dmenu-2780d184753f) which suggested just calling up a small terminal window and piping a list of applications to [fzf](https://github.com/junegunn/fzf), a blazing fast search tool. + +That's what I've done and it works great. I created a keybinding to launch a new instance of Alacritty with a class name that I use to resize the window. Then within that small Alacritty window I call `compgen` to get a list of executables, then sort it to eliminate duplicates, and pass the results to fzf. Here's the code in my Sway config file: + +~~~console +bindsym $mod+Space exec alacritty --class 'launcher' --command bash -c 'compgen -c | sort -u | fzf | xargs -r swaymsg -t command exec' + +for_window [app_id="^launcher$"] floating enable, border none, resize set width 25 ppt height 20 ppt, move position 0 px 0 px +~~~ + +These lines together will open a small terminal window in the upper left corner of the screen with a fzf search interface. I type, for example, "dar" and Darktable comes up. I hit return, the terminal window closes, and Darktable launches. It's as simple as dmenu and requires no extra applications (since I was already using fzf in Vim). + +##### Browsers + +I mainly use [qutebrower](https://qutebrowser.org/), supplemented by [Vivaldi](https://vivaldi.com/)[^2] for research because having split screen tabs is brilliant for research. I also use [Firefox Developer Edition](https://www.mozilla.org/en-US/firefox/developer/) for any web development work, because the Firefox dev tools are far superior to anything else. + +Qutebrowser and Firefox work great under Wayland. In the case of qutebrowser though you'll need to set a few shell variables to get it to start under Wayland, out of the box it launches with XWayland for some reason. Here's what I added to `.bashrc` to get it to work: + +~~~bash +export XDG_SESSION_TYPE=wayland +export GDK_BACKEND=wayland +~~~ + +One thing to bear in mind if you do have a lot of X11 apps still is that with this in your shell you'll need to reset the `GDK_BACKEND` to X11 or those apps won't launch. Instead you'll get an error, `Gtk-WARNING **: cannot open display: :0`. To fix that error you'll need to reset `GDK_BACKEND=x11`, then launch your X11 app. + +There are several ways you can do this, but I prefer to override apps in `~/bin` (which is on my $PATH). So, for example, I have a file named `vivaldi` in `~/bin` that looks like this: + +~~~bash +#! /bin/sh +GDK_BACKEND=x11 vivaldi-stable +~~~ + +Alternately you could just create a bash alias like `alias vivaldi-stable="GDK_BACKEND=x11 vivaldi-stable"`, but I prefer my `~/bin` method since I override a bunch of other apps too for other reasons and this way everything is in one spot. + +Note that for me this is easier, because the only apps I'm using that need X11 are Vivaldi and Slack. If you have a lot of X11 apps, you're probably better off making qutebrowser the special case by launching it like this: + +~~~bash +GDK_BACKEND=wayland qutebrowser +~~~ + +##### Clipboard + +I can't work without a clipboard manager, I keep the last 200 things I've copied, and I like to have things permanently stored as well. + +Clipman does a good job of saving clipboard history. + +You need to have wl-clipboard installed since Clipman reads and writes to and from that. I also use wofi instead of the default dmenu for viewing and searching clipboard history. Here's how I set up clipman in my Sway config file: + +~~~bash +exec wl-paste -t text --watch clipman store --max-items=60 --histpath="~/.local/share/clipman.json" +bindsym $mod+h exec clipman pick --tool="wofi" --max-items=30 --histpath="~/.local/share/clipman.json" +~~~ + +Clipman does not, however, have a way to permanently store bits of text. That's fine. Permanently stored bits of frequently used text are really not all that closely related to clipboard items and lumping them together in a single tool isn't a very Unix-y approach. Do one thing, do it well. + +For snippets I ended up bending [pet](https://github.com/knqyf263/pet), the "command line snippet manager" a little and combining it with the small launcher-style window idea above. So I store snippets in pet, mostly just `printf "my string of text"`, call up an Alacritty window, search, and hit return to inject the pet snippet into the clipboard. Then I paste it were I need it. + +##### Volume Controls + +Sway handles volume controls with pactl. Drop this in your Sway config file and you should be good: + +~~~bash +bindsym XF86AudioRaiseVolume exec pactl set-sink-volume @DEFAULT_SINK@ +5% +bindsym XF86AudioLowerVolume exec pactl set-sink-volume @DEFAULT_SINK@ -5% +bindsym XF86AudioMute exec pactl set-sink-mute @DEFAULT_SINK@ toggle +bindsym XF86AudioMicMute exec pactl set-source-mute @DEFAULT_SOURCE@ toggle +~~~ + +##### Brightness + +I like [light](https://github.com/haikarainen/light) for brightness. Once it's installed these lines from my Sway config file assign it to my brightness keys: + +~~~bash +bindsym --locked XF86MonBrightnessUp exec --no-startup-id light -A 10 +bindsym --locked XF86MonBrightnessDown exec --no-startup-id light -U 10 +~~~ + +### Quirks, Annoyances And Things I Haven't Fixed + +There have been surprisingly few of these, save the Vim and Ranger issues mentioned above. + +I haven't found a working replacement for xcape. The only thing I used xcape for was to make my Cap Lock key dual-function: press generates Esc, hold generates Control. So far I have not found a way to do this in Wayland. There is ostensibly [caps2esc](https://gitlab.com/interception/linux/plugins/caps2esc), but it's poorly documented and all I've been able to reliably do with it is crash Wayland. + +I miss xcape. I've reverted to `inoremap jj ` for vim, but if you have a solution, please do let me know in the comments. + +I've been using Waybar as a status bar and it works well enough. To be honest I haven't put in the time to get it exactly how I want it, but it's close enough. My one gripe is that Slack doesn't show up in the icon tray, but I hate Slack and only use it because of work, so my motivation to debug that issue is very low. + +Surprisingly, those are the only problems I can think of right now. I've been using Sway exclusively for almost two months now and I have no reason or desire to ever go back. + +### Helpful pages: + +- **[Sway Wiki](https://github.com/swaywm/sway/wiki)**: A good overview of Sway, config examples (how to replicate things from i3), and application replacement tips for i3 users (like this fork of [redshift](https://github.com/minus7/redshift/tree/wayland) with support for Wayland). +- **[Arch Wiki Sway Page](https://wiki.archlinux.org/index.php/Sway)**: Another good Sway resource with solutions to a lot of common stuff: set wallpaper, take screenshots, HiDPI, etc. +- **[Sway Reddit](https://old.reddit.com/r/swaywm/)**: There's some useful info here, worth searching if you run into issues. Also quite a few good tips and tricks from fellow Sway users with more experience. +- **[Drew Devault's Blog](https://drewdevault.com/)**: He doesn't always write about Sway, but he does give updates on what he's working on, which sometimes has details on Sway updates. + + +[^1]: That's not to imply there's anything wrong with the i3 developers. + +[^2]: Vivaldi would be another good example of me trusting a developer. I've been interviewing Jon von Tetzchner for many years, all the way back to when he was at Opera. I don't always see eye to eye with him (I wish Vivaldi were open source) but I trust him, so I use Vivaldi. It's the only software I use that's not open source (not including work, which requires quite a few closed source crap apps). diff --git a/src/src-ranger.txt b/src/src-ranger.txt new file mode 100644 index 0000000..f66a707 --- /dev/null +++ b/src/src-ranger.txt @@ -0,0 +1,63 @@ +[Ranger](http://nongnu.org/ranger/) is a terminal-based file browser with Vim-style keybindings. It uses ncurses and can hook into all sorts of other command line apps to create an incredibly powerful file manager. + +If you prefer a graphical experience, more power to you. I'm lazy. Since I'm already using the terminal for 90 percent of what I do, it make sense to not leave it just because I want to browse files. + +The keyword here for me is "browse." I do lots of things to files without using Ranger. Moving, copying, creating, things like that I tend to do directly with `cp`, `mv`, `touch`, `mkdir` and so on. But sometimes you want *browse* files, and in those cases Ranger is the best option I've used. + +That said, Ranger is something of a labyrinth of commands and keeping track of them all can be overwhelming. If I had a dollar for every time I've searched "show hidden files in Ranger" I could buy you a couple beers (the answer, fellow searchers, is `zh`). + +I'm going to assume you're familiar with the basics of movement in Ranger like `h`, `j`, `k`, `l`, `gg`, and `G`. Likewise that you're comfortable with `yy`, `dd`, `pp`, and other copy, cut, and paste commands. If you're not, if you're brand new to ranger, check out [the official documentation](https://github.com/ranger/ranger/wiki/Official-user-guide) which has a pretty good overview of how to do all the basic stuff you'll want to do with a file browser. + +Here's a few less obvious shortcuts I use all the time. Despite some overlap with Vim, I do not find these particularly intuitive, and had a difficult time remembering them at first: + +- `zh`: toggle hidden files +- `gh`: go home (`cd ~/`) +- `oc`: order by create date (newest at top) +- `7j`: jump down seven lines (any number followed by j or k will jump that many lines) +- `7G`: jump to line 7 (any number followed by `G` will jump to that line +- `.d`: show only directories +- `.f`: show only files +- `.c`: clear any filters (such as either of the previous two commands) + +Those are handy, but if you really want to speed up Ranger and bend it to the way you work, the config file is your friend. What follows are a few things I've done to tweak Ranger's config file to make my life easier. + +###Ranger Power User Recommendations + +Enabling line numbers was a revelation for me. Open `~/.config/ranger/rc.conf` and search for `set line_numbers` and change the value to either `absolute` or `relative`. The first numbers from the top no matter what, the `relative` option sets numbers relative to the cursor. I can't stand relative, but absolute works great for me, YMMV. + +Another big leap forward in my Ranger productivity came when I discovered local folder sorting options. As noted above, typing `oc` changes the sort order within a folder to sort by date created[^1]. While typing `oc` is pretty easy, there are some folders that I *always* want sorted by date modified. That's easily done with Ranger's `setlocal` config option. + +Here's a couple lines from my `rc.conf` file as an example: + +~~~bash +setlocal path=~/notes sort mtime +setlocal path=~/notes/reading sort mtime +~~~ + +This means that every time I open `~/notes` or `~/notes/reading` the files I've worked with most recently are at the top (and note that you can also use `sort_reverse` instead of `sort`). That puts the most recently edited files right at the top where I can find them. + +Having my most recent notes at the top of the pane is great, but what makes it even more useful is having line wrapped file previews so I don't even need to open the file to read it. To get that I currently use the latest Git version of Ranger which I installed via [Arch Linux's AUR](https://aur.archlinux.org/packages/ranger-git/). + +This feature, which is invaluable to me since one of my common use cases for Ranger is quickly scanning a bunch of text files, has been [merged to master](https://github.com/ranger/ranger/pull/1322), but not released yet. If you don't [use Arch Linux](/src/why-i-switched-arch-linux) you can always build from source, or you can wait for the next release which should include an option to line wrap your previews. + +###Bookmarks + +Part of what makes Ranger incredibly fast are bookmarks. With two keystrokes I can jump between folders, move/copy files, and so on. + +To set a bookmark, navigate to the directory, then hit `m` and whatever letter you want to serve as the bookmark. Once you've bookmarked it, type ``` to jump straight to that directory. I try to use Vim-like mnemonics for my bookmarks, e.g. ``d` takes me to documents, ``n` takes me to `~/notes`, ``l` takes me to the dev folder for this site, and so on. As with the other commands, typing just ``` will bring up a list of your bookmarks. + +###Conclusion + +Ranger is incredibly powerful and almost infinitely customizable. In fact I don't think I really appreciated how customizable it was until I wrote this and dug a little deeper into all the ways you can map shell scripts to one or two character shortcuts. It can end up being a lot to keep track of though. I suggest learning maybe one or two new shortcuts a week. When you know longer have to think abut them, move on to the next couple. + +Or you can do what I do, wait until you have something you want to do, but don't know how, figure out how to do it, then write it down so you remember it. + +####Shoulders Stood Upon + +* [Dquinton's Ranger setup details](http://dquinton.github.io/debian-install/config/ranger.html) - I have no idea who this person is, but their Ranger setup and detailed notes was hugely helpful. +* [Ranger documentation](https://ranger.github.io/ranger.1.html) - The docs have a pretty good overview of the options available, though sometimes it's challenging to translate that into real world use cases. +* [Arch Wiki Ranger page](https://wiki.archlinux.org/index.php/Ranger) - Where would we be without the Arch Wiki? + + + +[^1]: In fact, just type `o` and you'll get a list of other sorting options (and if you know what `normal` means, drop me a comment below, I'm still trying to figure out what that means). diff --git a/src/src-solving-common-nextcloud-problems.txt b/src/src-solving-common-nextcloud-problems.txt new file mode 100644 index 0000000..b32a629 --- /dev/null +++ b/src/src-solving-common-nextcloud-problems.txt @@ -0,0 +1,92 @@ +I love [NextCloud](https://nextcloud.com). Nextcloud allows me to have all the convenience of Dropbox, but hosted by me, controlled by me, and customized to suit my needs. I mainly use the file syncing, calendar, and contacts features, but Nextcloud can do a crazy amount of things. + +The problem with NextCloud, and maybe you could argue that this is the price you pay for the freedom and control, is that I find it requires a bit of maintenance to keep it running smoothly. Nextcloud does some decidedly odd things from time to time, and knowing how to deal with them can save you some disk space and maybe avoid syncing headaches. + +I should note, that while I call these problems, I **have never lost data** using Nextcloud. These are really more annoyances and some ways to prevent them than *problems*. + +### How to Get Rid of Huge Thumbnails in Nextcloud + +If Nextcloud is taking up more disk space than you think it should, or your Nextcloud storage space is just running low, the first thing to check is the image thumbnails directory. + +At one point I poked around in the Nextcloud `data` directory and found 11-gigabytes worth of image previews for only 6-gigabytes worth of actual images stored. That is crazy. That should never happen. + +Nextcloud's image thumbnail defaults err on the side of "make it look good in the browser" where as I prefer to err on the side of keep it really small. + +I did some research and came up with a few solutions. First, it looks like my runaway 11-gigabyte problem might have been due to a bug in older versions of Nextcloud. Ideally I will not hit that issue again. But, I don't admin servers with hope and optimism, so I figured out how to tell Nextcloud to generate smaller image previews. I almost never look at the images within the web UI, so I really don't care about the previews at all. I made them much, much smaller than the defaults. Here's the values I use: + +~~~bash +occ config:app:set previewgenerator squareSizes --value="32 256" +occ config:app:set previewgenerator widthSizes --value="256 384" +occ config:app:set previewgenerator heightSizes --value="256" +occ config:system:set preview_max_x --value 500 +occ config:system:set preview_max_y --value 500 +occ config:system:set jpeg_quality --value 60 +occ config:app:set preview jpeg_quality --value="60" +~~~ + +Just ssh into your Nextcloud server and run all these commands. If you followed the basic Nextcloud install instructions you'll want to run these as your web server user. For me, with NextCloud running on Debian 10, the full command looks like this: + +~~~bash +sudo -u www-data php /var/www/nextcloud/occ config:app:set previewgenerator squareSizes --value="32 256" +sudo -u www-data php /var/www/nextcloud/occ config:app:set previewgenerator widthSizes --value="256 384" +# and so on, running all the commands listed above +~~~ + +This assumes you installed Nextcloud into the directory `/var/www/nextcloud`, if you installed it somewhere else, adjust the path to the Nextcloud command line tool `occ`. + +That will stop Nextcloud from generating huge preview files. So far so good. I deleted the existing previews and reclaimed 11-gigabytes. Sweet. You can pre-generate previews, which will make the web UI faster if you browse images in it. I do not, so I didn't generate any previews ahead of time. + +### How to Solve `File is Locked` Issues in Nextcloud + +No matter what I do, I always end up with locked file syncing issues. Researching this led me to try using Redis to cache things, but that didn't help. I don't know why this happens. I blame PHP. When in doubt, blame PHP. + +Thankfully it doesn't happen very often, but every six months or so I'll see an error, then two, then they start piling up. Here's how to fix it. + +First, put Nextcloud in maintenance mode (again, assuming Debian 10, with Nextcloud in the `/var/www/nextcloud` directory): + +~~~bash +sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on +~~~ + +Now we're going directly into the database. For me that's Postgresql. If you use Mysql or Mariadb, you may need to adjust the syntax a little. + +~~~bash +psql -U yournextclouddbuser -hlocalhost -d yournextclouddbname +password: +nextclouddbname=> DELETE FROM oc_file_locks WHERE True; +~~~ + +That should get rid of all the locked file problems. For a while anyway. + +Don't forget to turn maintenance mode off: + +~~~bash +sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off +~~~ + +### Force a File Re-Scan + +If you frequently add and remove folders from Nextcloud, you may sometimes run into issues. I usually add a folder at the start of a new project, and then delete it when the project is finished. Mostly this just works, even with shared folders, on the rare occasion that I used them, but sometimes Nextcloud won't delete a folder. I have no idea why. It just throws an unhelpful error in the web admin and refuses to delete the folder from the server. + +I end up manually deleting it on the server using: `rm -rf path/to/storage/folder`. Nextcloud however, doesn't always seem to notice that the folder is gone, and still shows it in the web and sync client interfaces. The solution is to force Nextcloud to rescan all its files with this command: + +~~~bash +sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on +sudo -u www-data php /var/www/nextcloud/occ files:scan --path="yournextcloudusername/files/NameOfYourExternalStorage" +sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off +~~~ + +Beware that on large data directories this can take some time. It takes about 30 seconds to scan my roughly 30GB of files. + +### Mostly Though, Nextcloud is Awesome + +Those are three annoyances I've hit with Nextcloud over the years and the little tricks I've used to solve them. Lest anyone think I am complaining, I am not. Not really anyway. The image thumbnail thing is pretty egregious for a piece of software that aims to be enterprise grade, but mostly Nextcloud is pretty awesome. + +I rely on Nextcloud for files syncing, Calendar and Contact hosting, and keeping my notes synced across devices. Aside from these three things, I have never had a problem. + +####Shoulder's Stood Upon + +* [Nextcloud's documentation](https://docs.nextcloud.com) isn't the best, but can help get you pointed in the right direction. +* I tried a few different solutions to the thumbnail problem, especially helpful was this post on [Understanding and Improving Nextcloud Previews](https://ownyourbits.com/2019/06/29/understanding-and-improving-nextcloud-previews/), but nachoparker. +* The [file lock solution](https://help.nextcloud.com/t/file-is-locked-how-to-unlock/1883) comes from the Nextcloud forums. +* The solution to scanning external storages comes from the [Nextcloud forums](https://help.nextcloud.com/t/automate-occ-filescan/35282/4). diff --git a/src/src-why-i-built-my-own-mailing-list-software.txt b/src/src-why-i-built-my-own-mailing-list-software.txt new file mode 100644 index 0000000..b9877cf --- /dev/null +++ b/src/src-why-i-built-my-own-mailing-list-software.txt @@ -0,0 +1,52 @@ +This is not a tutorial. If you don't already know how to write the code you need to run a mailing list, you probably shouldn't try to do it yourself. Still, I wanted to outline the reasons I built my own mailing list software in 2020, when there are dozens of commercial and open source projects that I could have used. + +At the core of my otherwise questionable decision is the notion that we ought to completely understand the core infrastructures in our lives. Why? Because it adds value and meaning to your life in the form of understanding. And that understand doesn't stop with the thing you understand either, it becomes part of you, you will find other places this understanding helps you. + +It's also just not that hard to do things yourself. It makes maintaining the system easier, and it often saves time (or money) in the long term. + +The only way to really understand a thing is to either build it yourself from scratch or completely disassemble it and put it back together. + +This is true for software as well as the rest of the world. I ripped all the electrical, propane, plumbing, and engine systems out of my home ([a 1969 RV](/1969-dodge-travco-motorhome)) because I needed to know how every single piece works, and how they all work together. + +I understand those systems now because I built them myself (with expert help when needed), and that makes maintaining them much easier. Otherwise I would always be dependant on someone else to keep my home running and that's no way to live. + +The same is true with software. If the software you're considering is a core part of your personal or business infrastructure, you need to understand every single part of it and how all those parts fit together. + +The question is, should you deconstruct an existing project or write your own from scratch? The answer depends on the situation, the right choice won't always be the same in every case. I do a mix a both and I'm sure most other people do too. There's no one right answer, which means you have to think things through in detail ahead of time. + +When I decided I wanted to [start a mailing list](/jrnl/2020/11/invitation), I looked around at the software that was available and very quickly realized that I had different goals than most mailing list software. That's when you should write your own. + +The available commercial software did not respect users privacy and did not allow me any control. There are some services that do provide a modicum of privacy for your subscribers, but you're going to be working against the software to enable it. + +*If you know of a dead simple commercial mailing list software that's built with user privacy in mind, please post a link in the comments, I'd love to have somewhere to point people. * + +I also wanted to be in complete control of the data. I host my own publishing systems. I consider myself a writer first, but publisher is a close second. What sort of publisher doesn't control their own publishing system?[^1] What makes email such a wonderful distributed publishing system is that no one owns the protocols that dictate how it works. That's great. I don't want to control the delivery mechanism, just the product at either end. + +Email is more or less the inverse of the web. You send a single copy to many readers, rather than many readers coming to a single copy as with a web page. The point is, there's no reason I can't create and host the original email here and send out the copies myself. The hard part -- creating the protocols and low-level tools that power email -- was taken care of decades ago. + +With that goal in mind I started looking at open source solutions. I use [Django](https://www.djangoproject.com) to publish what you're reading here, so I looked at some Django-based mailing list software. The two I considered most seriously were [Django Newsletter](https://django-newsletter.readthedocs.io/en/latest/) and [Emencia Django Newsletter](https://github.com/emencia/emencia-django-newsletter). I found a few other smaller projects as well, but those seem to be the big two in what's left of the Django universe. + +Those two, and some others influenced what I ended up writing in various ways, but none of them were quite what I wanted out of the box. Most of them still used some kind of tracking, whether a pixel embedded in the email or wrapping links with individual identifiers. I didn't want either of those things and stripping them out, while staying up-to-date with upstream changes would have been cumbersome. So, DIY then. + +But running a mail server is... difficult, risky, and probably going to keep you up at night. I tried it, briefly. + +One of the big problems with email is that, despite email being an open protocol, Google and other big corps are able to gain some control by using spam as a reason to tightly control who gets to send email[^2] That means if I just spin up a VPS at Vultr and try to send some emails with Postfix they're probably all going to end up in, best case, you Spam folder, but more likely they'd never be delivered. + +So while I wrote the publishing tools myself, host the newletter archive myself, designed everything about it myself, I handed off the sending to [Amazon's SES](https://aws.amazon.com/ses/), which has been around long enough, and is used by enough big names that mail sent through it isn't automatically deleted. It may possibly still end up in some Spam folders, but for the most part in my early testing (thank you to all my friends who helped out with that) that hasn't been an issue. + +In the end what I have is a fairly robust, loosely-joined system where I have control over the key elements and it's easy to swap out the sending mechanism down the road should I have problems, or just find something better (preferably something not owned by Amazon). + +###Was it Worth It? + +So far absolutely not. But I knew that when I started. + +I could have signed up for Mailchimp, picked some pre-made template, and spent the last year sending out newsletters to subscribers, and who knows, maybe I'd have tons of those by now. But that's okay, that was never the goal. + +I am and always have been playing a very long game when it comes to publishing. I am building a thing that I want to last the rest of my life and beyond if I can manage it. + +I am patient. I am not looking for a ton of readers, I am looking for the right readers. The sort of people who are in short supply these days, the sort of people who end up on a piece like this and actually read the whole thing. The people for whom signing up for Mailchimp would be too easy, too boring. + +I am looking for those who want some adventure in everything they do, the DIYer, the curious, the explorers, the misfits. There's more of us than most of us realize. If you're interested feel free to [join our club](/newsletter/friends). + +[^1]: Sadly, these days almost no publisher retains any control over their systems. They're all beholden to Google AMP, Facebook News, and whatever the flavor of year happens to be. A few of them are slowly coming around to the idea that it might be better to build their own audiences, which somehow passed for revolutionary in publishing today. But I digress. +[^2]: Not to go too conspiracy theory here, but I suspect that Google and its ilk generate a fair bit of the spam themselves, and do nothing to prevent the rest precisely because it allows for this control. Which is not to say spam isn't a problem, just that it's a very *convenient* problem. diff --git a/src/why-i-built-my-own-mailing-list-software.txt b/src/why-i-built-my-own-mailing-list-software.txt deleted file mode 100644 index b9877cf..0000000 --- a/src/why-i-built-my-own-mailing-list-software.txt +++ /dev/null @@ -1,52 +0,0 @@ -This is not a tutorial. If you don't already know how to write the code you need to run a mailing list, you probably shouldn't try to do it yourself. Still, I wanted to outline the reasons I built my own mailing list software in 2020, when there are dozens of commercial and open source projects that I could have used. - -At the core of my otherwise questionable decision is the notion that we ought to completely understand the core infrastructures in our lives. Why? Because it adds value and meaning to your life in the form of understanding. And that understand doesn't stop with the thing you understand either, it becomes part of you, you will find other places this understanding helps you. - -It's also just not that hard to do things yourself. It makes maintaining the system easier, and it often saves time (or money) in the long term. - -The only way to really understand a thing is to either build it yourself from scratch or completely disassemble it and put it back together. - -This is true for software as well as the rest of the world. I ripped all the electrical, propane, plumbing, and engine systems out of my home ([a 1969 RV](/1969-dodge-travco-motorhome)) because I needed to know how every single piece works, and how they all work together. - -I understand those systems now because I built them myself (with expert help when needed), and that makes maintaining them much easier. Otherwise I would always be dependant on someone else to keep my home running and that's no way to live. - -The same is true with software. If the software you're considering is a core part of your personal or business infrastructure, you need to understand every single part of it and how all those parts fit together. - -The question is, should you deconstruct an existing project or write your own from scratch? The answer depends on the situation, the right choice won't always be the same in every case. I do a mix a both and I'm sure most other people do too. There's no one right answer, which means you have to think things through in detail ahead of time. - -When I decided I wanted to [start a mailing list](/jrnl/2020/11/invitation), I looked around at the software that was available and very quickly realized that I had different goals than most mailing list software. That's when you should write your own. - -The available commercial software did not respect users privacy and did not allow me any control. There are some services that do provide a modicum of privacy for your subscribers, but you're going to be working against the software to enable it. - -*If you know of a dead simple commercial mailing list software that's built with user privacy in mind, please post a link in the comments, I'd love to have somewhere to point people. * - -I also wanted to be in complete control of the data. I host my own publishing systems. I consider myself a writer first, but publisher is a close second. What sort of publisher doesn't control their own publishing system?[^1] What makes email such a wonderful distributed publishing system is that no one owns the protocols that dictate how it works. That's great. I don't want to control the delivery mechanism, just the product at either end. - -Email is more or less the inverse of the web. You send a single copy to many readers, rather than many readers coming to a single copy as with a web page. The point is, there's no reason I can't create and host the original email here and send out the copies myself. The hard part -- creating the protocols and low-level tools that power email -- was taken care of decades ago. - -With that goal in mind I started looking at open source solutions. I use [Django](https://www.djangoproject.com) to publish what you're reading here, so I looked at some Django-based mailing list software. The two I considered most seriously were [Django Newsletter](https://django-newsletter.readthedocs.io/en/latest/) and [Emencia Django Newsletter](https://github.com/emencia/emencia-django-newsletter). I found a few other smaller projects as well, but those seem to be the big two in what's left of the Django universe. - -Those two, and some others influenced what I ended up writing in various ways, but none of them were quite what I wanted out of the box. Most of them still used some kind of tracking, whether a pixel embedded in the email or wrapping links with individual identifiers. I didn't want either of those things and stripping them out, while staying up-to-date with upstream changes would have been cumbersome. So, DIY then. - -But running a mail server is... difficult, risky, and probably going to keep you up at night. I tried it, briefly. - -One of the big problems with email is that, despite email being an open protocol, Google and other big corps are able to gain some control by using spam as a reason to tightly control who gets to send email[^2] That means if I just spin up a VPS at Vultr and try to send some emails with Postfix they're probably all going to end up in, best case, you Spam folder, but more likely they'd never be delivered. - -So while I wrote the publishing tools myself, host the newletter archive myself, designed everything about it myself, I handed off the sending to [Amazon's SES](https://aws.amazon.com/ses/), which has been around long enough, and is used by enough big names that mail sent through it isn't automatically deleted. It may possibly still end up in some Spam folders, but for the most part in my early testing (thank you to all my friends who helped out with that) that hasn't been an issue. - -In the end what I have is a fairly robust, loosely-joined system where I have control over the key elements and it's easy to swap out the sending mechanism down the road should I have problems, or just find something better (preferably something not owned by Amazon). - -###Was it Worth It? - -So far absolutely not. But I knew that when I started. - -I could have signed up for Mailchimp, picked some pre-made template, and spent the last year sending out newsletters to subscribers, and who knows, maybe I'd have tons of those by now. But that's okay, that was never the goal. - -I am and always have been playing a very long game when it comes to publishing. I am building a thing that I want to last the rest of my life and beyond if I can manage it. - -I am patient. I am not looking for a ton of readers, I am looking for the right readers. The sort of people who are in short supply these days, the sort of people who end up on a piece like this and actually read the whole thing. The people for whom signing up for Mailchimp would be too easy, too boring. - -I am looking for those who want some adventure in everything they do, the DIYer, the curious, the explorers, the misfits. There's more of us than most of us realize. If you're interested feel free to [join our club](/newsletter/friends). - -[^1]: Sadly, these days almost no publisher retains any control over their systems. They're all beholden to Google AMP, Facebook News, and whatever the flavor of year happens to be. A few of them are slowly coming around to the idea that it might be better to build their own audiences, which somehow passed for revolutionary in publishing today. But I digress. -[^2]: Not to go too conspiracy theory here, but I suspect that Google and its ilk generate a fair bit of the spam themselves, and do nothing to prevent the rest precisely because it allows for this control. Which is not to say spam isn't a problem, just that it's a very *convenient* problem. -- cgit v1.2.3-70-g09d2