Overwrite/Restore options for Sync in Codebook 4

Likewise Overwrite/Restore allows me to do similar to jtrh with Onedrive as my base. I then have phone, desktop and laptop synced to a common database on Onedrive.

1 Like

I agree … we have 8 different devices I work on with my codebook database (iPhones, iPads, windows laptops, and desktop). So far all have worked absolutely perfectly (love this software).
Depending on which device I use, I find I need to restore or overwrite often. Losing this option worries me for the same reason as this poster. I am not confident that one device updated And syncd will then cancel out if another sync is made on one of my other devices before some other change is made along the way. Knowing I can force everyone to either overwrite or restore gives me the confidence that I’m sure the latest sync will be made to the database and I won’t lose anything. I vote to keep the option to overwrite/restore with the sync as well.

1 Like

Previously with Codebook 3 all Sync Data was encrypted with your master password, and stored as one giant database out on the remote (Dropbox, Google Drive, or Desktop via Wifi). On the one hand this was simple and made functions like Overwrite and Restore possible. This was very secure and worked well initially but there were many problems with this approach that became apparent over time. Besides being slow and inefficient, it prevented us from implementing many new features that customers have asked for like storage of images and attachments, multi-factor authentication, and fast background sync. These are all future plans, so we needed to replace the sync system.

The new sync approach with Codebook 4 is quite different. Obviously the Sync Key is new and very visible. But the more significant change is that sync changes are broken up into small chunks that describe individual changes. These chunks are stored as a chronological series of change sets. This means that now Codebook only needs to exchange the items that have changed when it syncs, instead of copying your entire encrypted database around. The use of the random Sync Key makes it even more secure than before.

However, this change also means that the old concepts of Overwrite and Restore, where database were just deleted and replaced, are no longer applicable. For example, it isn’t feasible to overwrite in the new model, because changes have been tracked in such a way that Codebook can’t replace all the remote data. It is for this reason that we don’t foresee the return of Overwrite and Restore functions as first class sync features.

We have gone to great lengths to ensure that even when you are syncing across multiple devices changes will be merged in a sensible manner. In other words, the most recent change will always be visible after sync, new entries and categories will be created across your devices, and deletions will propagate cleanly. Our intent is that you should be able to use the new sync system without having to worry at all about what device a change originated on. Sync will be much faster too, especially over slower data connections. Furthermore, the new system will support future features like attachments, multi-factor authentication, and background sync seamlessly.

That said, we do recognize that in some cases it is required to completely replace a local dataset and start from a good copy, for example in a data recovery scenario. If you have bad data in a local copy (e.g. you have accidentally deleted an entire category of entries), you can uninstall the app on mobile, or delete your Codebook application data folders on desktop, reinstall, and start over. The next time you sync that “new” copy it will restore the good data from your Cloud or Desktop system.

We’d very much like for you to try out the new system with bidirectional sync support and see if it works for you. Of course we are grateful to hear from users, and appreciate feedback. If after using the new sync system you still find there there are specific scenarios or use cases that you feel can’t be supported we’d be happy to hear about them.

Okay your arguments are reasonable and make perfect sense. The idea of updating only what’s changed is also a good idea. I am trying now across my 8devices and so far no problems :yum:

Georgia Killcrece

And, just how do we delete the “Codebook application data folders” on a Mac desktop? You say “reinstall” does that mean remove the app bundle as well?

Hello @JFS - you can use these instructions to reset Codebook if you need it:

So what you are saying is we have to go through a convoluted process to overcome known and demonstrated shortcomings in your sync technology every time we’re worried about this. You’ve also removed local backups from the desktops, which leaves us no recourse.

Like many that use this product, I have my entire electronic life depending on this product. The fact that you handwave away these concerns without considering how solutions could be implemented is deeply disturbing. Based on the reliability I’ve experienced with the current sync option, I’m concerned about the future of the product.

I’ve never even contemplated this before now, but if I can’t trust Codebook, because I have no idea what it will do, I may have to find another product. I can’t afford to lose unrecoverable information due to the sync making poor decisions with old databases.

Please don’t half-ass this. See if you can find a way to give us more control over the integrity of our databases.

3 Likes

I share bobmhac’s concerns. I have quite a few times in the past had to use the restore option to recover something happening to a local database. I’m not yet convinced that these potential future features outweigh the protection of a true backup and restore feature.

Having second factor totp options and other digital information stored now makes me far more nervous about the integrity and maintenance of these databases. And I am fully also including the source of the disruption of the database to be user caused not just some sync of software bug.

Zetetic has also mentioned that this sync change lending to the possibility of some type of auto sync feature being possible in a future revision. This makes me even more nervous with no cloud back up and restore capability.

If we can’t have overwrite/restore to deal with unwanted changes, mistakes, sync conflicts, and so on, then it would be nice to have a per-entry version history for entries that are updated, and a recycle bin for entries that are deleted.

I was initially very enthused about the added security, speed and simplicity of the v4 design.

Please note the following recap may not be 100% accurate because it was written postmortem. Soon after a v4 upgrade, I discovered database labeling was not correct. I attempted a label correction on a PC hoping it would fix an autofill issue but it did not have the expected result. To revert the change, I exported a “good” database to a CSV file from a different PC and imported to the PC having the autofill problem. The import doubled the number of database entries with duplicates. Oops. I deleted everything and imported again. The PC looked ok. I then tried to sync to an IOS device and the number of entries doubled. I deleted all the categories on the IOS device and tried to sync but of course that wiped out the central repository. I did another delete / import / sync on the PC to “freshen” the shared repository on the gdrive. An IOS sync pulled the correct number of entries but several labels were now listed as “unknown”. I decided to give up on v4.

What I learned was a software bug and / or user errors can quickly result in corruption or data loss. V3 options to sync / overwrite / restore add resiliency. I did successfully revert to v3. Someday I look forward to upgrading to v4 but as it stands now, the risk from complications is too high.

Just tried doing the V4 Sync key thing, and now my database has no labels. Please bring back overwrite/restore. How are the labels getting stripped in the first place?

1 Like

@berlee This may have been an issue due to a discrepancy between the state of your Codebook 3 databases prior to upgrade. Please contact us directly at support@zetetic.net and we will work with you.

I would like to thank everyone on this thread for the candid feedback on the new Sync and the change to Restore / Overwrite (as well as the folks who have contacted us privately about this outside of the discussion forums).

We have received a large amount of feedback about this, including a number of valid use cases and scenarios about how Restore / Overwrite features were being used. We have learned a lot about the implications of the changes over the past couple of weeks.

After the holiday, we will be discussing feedback from the Codebook 4 release in an effort to guide our path for upcoming releases in the new year. As part of this we are definitely going to review the feedback about the Sync changes specifically, and all of this will be considered going forward.

1 Like

Sorry but I m not interested in the potentialy new functionalities you are mentioning (to save images or attachments or other stuff). I don’t trust your sync method anymore since it is not transparent at all…
Doing such major changes in the philosophy of your product without asking your users and the oldest ones is not professional and doesn’t make sense…
Thanks

1 Like

Well, I’m now having difficulty reliably syncing changes between two devices. Occasionally it’ll just forget that an update occurred and not push it to the shards. Only way for me to force that is to delete the shards folder on Dropbox and force a full sync; which is what I wanted to do in the first place.

I know you guys worked really hard on this, but it’s just go too many problems. Please add a preference option to use to old, slower method of syncing (if that can be done with the new encryption system). I’d rather wait a few more seconds than deal with integrity problems.

At a minimum, a button to “clear” the cloud storage could help, so I don’t have to go outside the program to accomplish the task.

1 Like

I have to second this. Overwrite and Restore actions are vital. I’m up to rev with 4.0.1 on MacOs, and 4.0.3 on IOS, and this new sync facility is still a real stinker. It can’t be trusted: modifications don’t always sync; newly added entries will not always appear on the synced device; deletions don’t always propagate. Once Codebook decides to drop an entry, no amount of editing that entry will get that entry synced to other devices. The only alternative is to manually delete the app and reinstall Codebook. What a mess. I’m open to exploring other password managers, does anyone have any suggestions?

@JFS

We appreciate the additional feedback. We’ve responded to your separate discussion forum post: Codebook 4 Sync SNAFUs and definitely would like to work with you to resolve this issue via our support email.

Still having intermittent problems with Sync not functioning reliably. This time it was between Windows 4.0.12.0 and iOS 4.0.4 (870). Updating on the Windows client was not pushing changes that the iOS client saw. I got it to work this time by making changes to that same field on the iOS side, and then pushing it back and forth.

Also, when interacting with significantly out of date clients, I’ve taken to a convoluted workflow:
-clearing the shards on Dropbox
-doing a sync with a trusted vault to populate Dropbox with a fresh database
-syncing the out-of-date client
-removing whatever Frankenstein monster may have been created in Dropbox
-syncing to Dropbox with a trusted vault again.

I know you guys worked really hard to implement a bunch of interesting features with the new sync, but at the end of the day it has one job to do and it seems to have some problems doing it. I really wish there was a way to implement a legacy mode that reduced the feature set and gave us more control over what sync is doing.

Hello @bobmhac - I’m sorry to hear about the continued trouble here. I wanted to let you know that we are currently working on some changes that will provide some advanced controls for the new sync system. These changes will provide a way to more directly control how the sync is working to resolve sync issues.

Version 4.1; Testers Wanted

TL;DR: We are preparing Codebook version 4.1 and looking for customers who want to get an early look at the new Operations features and provide feedback.

Hi folks,

Since early January we’ve been working on version 4.1, an update to all four of the Codebook apps (Android, iOS, macOS, and Windows) that is intended to do a few things:

  • Fix several edge case bugs in the Sync system
  • Provide better error handling and recovery during Sync
  • Introduce new Advanced options for changing how Sync operates

The new sync options were created in response to feedback we received here and in the support mailbox, as well as a need to have more flexibility when helping customers address bad data and other support scenarios. They are analogous to the previous Operations feature in Codebook 3 (Sync, Overwrite, and Restore) along with a new operation called Merge.

We’re currently testing 4.1 internally here at Zetetic. We will begin external beta testing once we’re fully satisfied with our internal testing, but in the meantime we were wondering if anybody here would like to get an earlier look at the new sync operations and provide feedback? Please fill out this short Google form and we’ll be in touch as soon as we’re ready to get started.

Thanks in advance to anyone with the time to test, and thanks for all the feedback here, we really appreciate it! Hopefully this message finds you safe and healthy wherever you are.