User Interface - SyncAdaptor - Local DB/FS - Some Custom Sync Mechanism

[linonetwo]

graph TB
  subgraph UserInterface1
    UI1["TW in TidGi-Mobile React Native WebView"]
  end
  
  subgraph SyncAdaptor1
    SA1["Android-FS-SyncAdaptor"]
  end
  
  subgraph LocalDBFS
    Local["Local SQLite DB + FS"]
  end
  
  subgraph CustomSyncMechanism
    Custom["Background task on android app"]
  end
  
  subgraph OtherClientServerDBFS
    Other["TidGi Desktop NodeJS Server's memory"] --> OtherFS["Desktop /tiddlers folder"]
  end

  subgraph CustomSyncMechanism2
    Custom2["Git-Sync And Github-Desktop"]
  end

  subgraph OtherClientServerDBFS2
    Other2["TidGi Desktop NodeJS Server's memory"]
    OtherFS2["Desktop /tiddlers folder"]
    OtherFS2 --> Other2
  end
  
  subgraph SyncAdaptor2
    SA2["NodeJS TiddlyWeb Sync Adaptor"]
  end
  
  subgraph UserInterface2
    UI2["TW in MacOS's TidGi-Desktop Electron BrowserView"]
  end

  subgraph UserInterface3
    UI3["Another PC's Electron BrowserView"]
  end
  
  UI1 --> SA1
  SA1 --> Local
  Local --> Custom
  Custom --> Other
  Other --> SA2
  SA2 --> UI2
  OtherFS --> Custom2
  Custom2 --> OtherFS2
  Other2 --> UI3

Loading

This is the TiddlyWiki sync architecture I'm using, which involves CustomSyncMechanism like "Background task on android app" and "Git-Sync And Github-Desktop".

To make Tiddlywiki Local-First, I found Sync Adaptor can only be configured to save changes to the FileSystem/SqliteDB/IndexedDB. But we still want to sync to another computer or device or cloud server, so a 3rd-party sync mechanism needs to be introduced.

• If we can allow two syncAdaptors to work at the same time, maybe we can make this process clearer by making everything ("Background task on android app" and "Git-Sync And Github-Desktop") a syncAdaptor.
  • Background sync adaptor can just dispatch sync tasks to an Android background task via IPC/PostMessage, and let native code do the HTTP request on the background (Because after 30 min, WebView may go to background and paused, but native code can still be running)
  • Git sync adaptor can call HTTP API of remote Git server to batch putTiddler
• Some servers require a denounced update, for example, Github, we might need the syncAdaptor to wait 30 min to perform a batch commit
  • Each client needs an operation_log database (e.g. in indexed db) to record deletion, otherwise, there is nowhere to know what was deleted after 30 minutes.
  • Also need to save modifications in the operation_log database, because 30 minutes usually involves a refresh of the page or a full restart.
• After 30min or offline for a while, there will be conflicts, so need to consider Revision field, who is using this? Should I use it? #7719

[Jermolene]

@linonetwo the limitation of a single sync adaptor isn't too restrictive in practice, because there's nothing that is special about a syncadaptor that cannot be achieved with a regular startup module that implements its own synchronisation logic.

[linonetwo]

That is what I'm currently doing, but a limited API will prevent bugs.

I also hope there can be a standard UI in wiki to control the sync adaptors, if there is no standard, gathering information from many different sync implementations will be difficult. So what I'm doing are just workarounds.