I suggest saving the files as text/markdown and sending them to your children as an email zip file. You could also print a book and give copies to your children and grandchildren.
“Only wimps use tape backup. REAL men just upload their important stuff on ftp and let the rest of the world mirror it.”
- Linus Torvalds
When we go, our children will throw almost everything away. And this is ok.
Information regularly moves storage devices or dies. If your work is not already published / disseminated by the public then it will disappear.
I built a go caldav server and client for my task management app (http://calmtasks.com) and had a similar experience, which surprised me. Go generally has at least one good, working, and well documented implementation for all standard protocols.
Apple calendar supports caldav but in a way not specified in the spec. I basically had to send requests and responses to figure out how it works. I would be willing to open source my server and client (alot of which was built using/on top of existing libraries) if there is interest.
The nuance is that Amazon updated their sdk so that the default is that the sdk no longer works for third party services that do not use the new checksum. There is a new configuration option in the sdk that either:
(1) requires the checksum (and breaks s3 usage for unsupported services); or
(2) only uses the checksum if the s3 service supports it.
The sdk default is (1) when this issue could have been avoided if the sdk default was (2).
Agree with all the comments that Amazon has never said or even implied that updates to their sdk would work on other s3 compatible services. However, the reality is that s3 has become a defacto standard and (unless this is a deliberate Amazon strategy) it would be relatively easy for Amazon to set this default that allows for but does not require changes to s3 compatible services or, if possible, to loudly flag these types of changes well in advance so that they don’t surprise users.
Great question! Most of the articles I read only focus on apps with multiple millions of users, but most software has a much smaller user base.
I assume that most things I build will only be used by me and a small group of users, and sometimes will also eventually be used by others. I generally only build tools that I use every day. For example:
- iCloud/Dropbox alternative macOS, iOS, and web app
- task management GTD macOS, iOS, and web app
- photos api for digital photo frames
- media management
Mostly built using go for backends and SwiftUI for macOS and iOS frontends.
For most web apps I like magic link over email with cookie sessions that expire after 30 days. JWT for apis authorised by email magic links. No passwords to be hacked. No third party dependencies.
I think that you can’t build it in a weekend. Well I couldn’t. As soon as you try to rapidly and reliably sync, say, 20k+ files over 3+ clients with intermittent internet connections, it gets surprisingly complicated.
That’s a rather specific edge case and I can’t think of any sync that handles it relatively well except for BitTorrent. At that point you are limited by so many things, it’s not worth optimizing for because all the error cases you have to handle (inode exhaustion, network limits, etc).
99.99% of the time only a single file changes at a time, which is the use case here.
Is it really an edge case these days? A user’s very first sync is likely to be the largest one, to seed all the data into the could. That could easily be thousands of files.
Installing on a desktop, laptop, and mobile device would bring 3 clients into the mix right away.
So the user’s first use is likely to be one of the more complex operations, and the one they absolutely needs to work, as a failure here will lead to the user looking elsewhere for a solution.
> A user’s very first sync is likely to be the largest one, to seed all the data into the could.
You are almost guaranteed that the first sync results in zero conflicts. It's just a mass-upload. Even if you bring other clients in, there is still a very low chance and "last-one-wins" is a pretty decent policy for who gets the original filename and then other gets a " (copy)" suffix or something. Doesn't really matter as long as it's documented and intuitive.
My point is that for a weekend PoC that you want to use for yourself… it’s rather a non-issue. Further, you need to create a lot of primitives before you can even tackle the problem correctly, so initially, it is an unsolvable problem until you have those primitives.
Hmm the macOS photos folders have this many files so I’m not sure it’s an edge case to sync this many files between at least 2 clients. And the issues you mention are real even when syncing relatively static large folders.
But you’re right that there are many such error cases to handle if you want a real iCloud / Dropbox alternative.
It was a lot of work but now I can sync multiple folders at once on macOS, not use the macOS file provider api, sync the macOS photos folder, access my synced files via an api, access file metadata (longitude and latitude so I can display photos from particular locations on a photo frame), listen to music in folders on the ios app, automatically transcode and watch uploaded videos on the ios app, etc, etc.
I’m using it myself and for a few others in private beta. My point is that sync done well can be done, it’s just that no-one seems to be providing the service.
“Only wimps use tape backup. REAL men just upload their important stuff on ftp and let the rest of the world mirror it.” - Linus Torvalds
When we go, our children will throw almost everything away. And this is ok.
Information regularly moves storage devices or dies. If your work is not already published / disseminated by the public then it will disappear.
And this is ok.