> At midnight UTC, the entire current month is refetched from the source as a si... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		palmotea 29 days ago \| parent \| context \| favorite \| on: Show HN: Hacker News archive (47M+ items, 11.6GB) ... > At midnight UTC, the entire current month is refetched from the source as a single authoritative Parquet file, and today's individual 5-minute blocks are removed from the today/ directory. Wouldn't that lose deleted/moderated comments?

BoredPositron 28 days ago [–]

I guess that's the point.

Imustaskforhelp 28 days ago | [–]

Can't someone create an automatic script which can just copy the files say 5 minutes before midnight UTC?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact