Hey @Dan_Danz
Glad to hear the explanation from the documentation was helpful.
I’d like to mention that aside from the first couple changeset files within the Shards directory (which contain entire replays of your databases), all the changeset files are very small, so even though the actual file counts are continually growing each time you sync, the actual sizes are very small. We like to think of the changesets required to reconstitute your database as growing reasonably and predictably. They allow us to bring any stale device up-to-date while also being able to set up a new device.
We always appreciate hearing suggestions from Codebook users, but unfortunately your suggestion wouldn’t work. Specifically we need to preserve the entire history of the Shards directory in case a new device is added and needs to apply the entirety of its changes. Each Codebook client is already tracking the “High Water Mark” of what it’s seen and what’s been seen by this specific remote, ensuring that it’s as efficient as possible and only downloading/applying/uploading changes that are unseen.
We may investigate providing an automated way of “consolidating” these changes in the future, but at this time because the file sizes are so small we haven’t seen the need.
That being said, as @wgray mentioned, in the next release of Codebook, 4.1, there will be an easy way for a user to clear out and re-establish their remote by utilizing the overwrite operation if they so desire.
We’ve got some more information in this other post that may help to put the design of the new sync system into perspective.