Clickhouse: Import compressed JSON fast

As part of the MVG Observatory Project I collect departures of the Munich public transport. Top level is the date and the subfolder includes multiple stations (with their id as the folder name). Each station includes lots of single files. Every day the top level folder gets archived into a zstd compressed file. To analyse the data, the content has to be imported into Clickhouse. 20240615/ ├── de:09162:1 │ ├── 1718409659_body....

October 21, 2024 · 4 min · 752 words · Robert

Fly with Sensible Photo Equipment

I got dragged into the rabbit hole of taking analog photos about a year ago and enjoying it since. But when you want to board a plane with your camera and a bunch of film rolls you might ask yourself: What to do with it during the x-ray at the security check. Will it harm or destroy my film? Film condition This concern is valid for a film that is either new or exposed and winded back which therefore is unprocessed....

April 22, 2024 · 5 min · 937 words · Robert

TIL: 'dot_clean' in macOS

When switching between multiple systems including macOS you may have notices thoses strange looking files starting with ._. They regularely break Switch firmware upgrades and might be in the way during other filesystem operations as well. There are multiple solutions out there in the wild including removing them or moving them away using rsync. The most convinient way on macOS is using the dot_clean command which is shipped with macOS by default....

April 7, 2023 · 1 min · 103 words · Robert

Advent of Code - RocketLang Edition

This years Advent of Code is a little special because I use my own language RocketLang to solve the puzzles. I’ll probably explain this in a seperate post at some point so lets have a look at the first puzzle of the year: Problem The task is to calculate the amount of calories they are carrying. For example: 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 This shows us 5 elves with different amount of items (with different calories each)....

December 1, 2022 · 2 min · 394 words · Robert

🪴 Blumentopf - Automated Plant Watering #2

This post misses a lot of technical details as I reconstruct the changes and implementations out of my memory and my image archive. The latest Blumentopf iteration will contain all technical details you need. In 2020 I remembered a Self-Watering Planter I printed a few years ago and thought it would be nice to reuse this one to replace my chunky prototype with a cleaner and more aesthetic version. Since I grew a lot of chilis that needed a similiar amount of water, I thought it would be easier to only use one pump and chain the pots together....

May 1, 2022 · 2 min · 401 words · Robert