Clickhouse: Import compressed JSON fast
As part of the MVG Observatory Project I collect departures of the Munich public transport. Top level is the date and the subfolder includes multiple stations (with their id as the folder name). Each station includes lots of single files. Every day the top level folder gets archived into a zstd compressed file. To analyse the data, the content has to be imported into Clickhouse. 20240615/ ├── de:09162:1 │ ├── 1718409659_body....