[SailfishDevel] SyncML Plugin Server question

deloptes deloptes at gmail.com
Wed Aug 7 11:33:05 UTC 2019

Chris Adams wrote:

> Thanks for doing that work!

And I thank you for your opinion and support so far, really appreciated as
the topic is very important (I hope not only for me). 

This message will be a bit longer, but I try to summarize here the most
important results and ideas that emerged out of this (poc) work so far. So
thank you for patiently reading and contributing to this subject. Also note
we are not in hurry as I will be off until end of August starting end of
this week.

Note: I use production or prod device to refer to the licensed phone I am
using as primary device ATM. There is a test phone I bought cheep and I use
for testing and refer to as test device, which is not licensed.

> I suspect that you are correct regarding the client-initiated syncs (in
> this use-mode, each Sailfish OS device assumes that it is both a SyncML
> server and a SyncML client, insofar as any change to its database might
> trigger a sync with another device (i.e. this device acts as a client) and
> any change to the other device's database might trigger it to sync with
> this device (i.e. this device acts as a server)).

I am not sure if this applies directly to the SyncML - when SyncML session
starts it does synchronization - in the process it finds out if there are
changes. What do you mean that changes on one end trigger the sync? I have
no changes and it still tries to sync.

Also consider following:

I did not catch this problem it on the test device, because there I did not
configure any account. Obviously if account is configured for
synchronization through the other profiles the sync is triggered as well.
So this is very interesting why if there is no account configured the BT
sync is not triggered. I think there is a missing link. In Nokia N9 in
Settings there is Sync & Backup and one can configure a Sync Profile for BT
devices. I think this is missing. So from the logs I see that when I
trigger the sync from Syncevolution on the PC the profile is being
generated/written to disc with setting enabled. The server is not stopped,
because of the original question I had and the finding that this is being
disabled after first sync was performed (reference to profile not removed).
So the server keeps the profile and the client tries to use the same profile
to initiate the sync. 
In the case of Syncevolution it does not work, because there is no server
actively running on the other side. 
I managed to narrow down and fix few things in the client plugin, so that it
does not crash and/or block the server profile. I'm still working on it and
will also tag you after I push to my PoC.

Two conclusions here:
1. there is something missing to configure the profile for a BT device
2. In general it is a good feature, because if two devices see each other,
they could sync automatically. The question is how do you tell the one to
be the client and the other to be the server. Should there be a
configuration option somewhere, that is missing or I did not see.

> This was the fundamental reason why the original calendar sync PR wasn't
> able to be merged: the semantics of such "round trip, multiple device"
> synchronisation cycles weren't well defined, and instead the sync acted
> more as an "import" than a true "sync" (potentially resulting in data
> duplication).

In all my life I had the only use case, where I used the PC as server and
the Phone as client. And I also had only 1 PC and 1 Phone. I am working now
slowly to get another device (perhaps Nokia N9) and test device to device
sync, but when I tried few days ago I did not see SyncML under profiles
supported by the dev X to appear in N9, however on the prod X I see it. 
It should be definitely a primary use case for this technology.

> (For example: what happens if you sync contacts 5 times between the
> devices?  If you get 5x the whole addressbook, resulting in 5 constituent
> contacts being linked into a single aggregate contact, then that is
> clearly suboptimal / should be fixed before we merge.)

SyncML should solve this problem. Well, from time to time it fails to solve
it completely (see further down slow-sync), but the use cases are well
defined and described. Don't get the wrong impression. I am not an expert
in SyncML, but I read some documentation in the past and I know that this
case is covered.
In Syncevolution it merges the information difference in the one account it
identifies - no duplicates of accounts. Only if it can not identify the
account, it would add a new one. In Sailfish, I need to find out how it is
done. AFAIR there is the option to configure and pass to syncml engine -
what action is to be taken when duplicates are identified: Server wins,
Client wins, Ask user -> user can merge or choose which record is to be
used. I am not 100% sure how it is - my memory is like 7-8y old.

>From what I read the concept of Buteo is that there would be 1 device - 1
Calendar/Contacts relation. This is also the case with Syncevolution and
AFAIK it is coming from SyncML. But I do not want to go into detail here.
It means you can define 1 calendar and 1 address book to sync with 1 other
device in 1 profile. You can create another profile to sync different set
of PIM.

> In many cases, what you want is actually a simple "import" from another
> device - but sharing the .vcf or .ics via Bluetooth sharing and then
> manually importing that via Settings->Apps->People->Import vCard is
> probably the better option.

I disagree here - the import is creating new entries and will overwrite or
create duplicates. The syncml however is importing/creating new if the item
was not found or updating the item by comparing the revision and in case of
slow sync the content or whatever - I do not have deeper knowledge. It is
important to understand that the content is compared and action is taken as
explained above. If I recall correctly this one was the reasons why
opensync died.

This makes slow sync most problematic or what people experience when they
first try syncing their Contacts and Calendar, which hold information on
both devices. First sync is always a slow sync and if you have similar data
(for example you exported/imported when switching devices), you get more
Alternatively I usually create empty Contacts/Calendar and do so my first
sync which means I import.

Nevertheless I tested actually both few days ago with ca. 250 contacts and
160 Calendar entries. In the problematic case where I had data on both ends
Calendars merged perfectly (is understandable). The contacts were compared
and merged. Only in 2 cases I had a duplicates where I intentionally
created same contact.

Do not understand me wrong. The solution is not perfect and needs more
attention before considered for mass production. I only want to highlight
here that there are different use cases that need to be taken into account.

So I do not know how to proceed from here. IMO we should sit down take the
whole picture and look where things need to be changed and are acceptable
from design and technology POC. I do not have the time to do and test
changes that would be rejected later, because not acceptable from higher
design or of other consideration. Jolla should know the big picture and the
community should fit the best.
I see Jolla as pretty flexible, but we are talking here about core
components that need some adjustments. This is why I stick so far with my
changes to the source as far as possible. So here are the questions and
suggestions that emerged from the PoC

1. buteo-syncfw 
        - it is not clear how client/server sync should work 
                - between two buteo snycml devices
                - between buteo snycml and other snycml devices (Syncevolution)
        - BT sync is not triggered automatically if no account is configured
        - BT sync is triggered automatically if one account is configured
        - there is no option to manage the configuration - perhaps under accounts

2. buteo-sync-plugins
        - I would move the bluez5 manager to the syncml server, but even this 
                is not really needed as it is managed by TransportTracker
        - using KF5BluezQt is acceptable if the socket issues can be solved
                so far I do not see any advantage using it except in the process 
                of initialization, which is also problematic as explained elsewhere
        - The old code handling the file descriptors should be replaced 
                with one suitable for use with bluez5 > 5.46
                but I do not see how it can be done with KF5BluezQt 
        - I am considering a new PoC with QtBluetooth or Qt DBus, but
                as the holidays are over, it might be taking time until next window

3. GUI integration
        - there should be an option like in N9 to configure the profile
                - sync direction etc. like in the Accounts 
                - the components (Calendar/Phonebook etc.)
                - automatically or not
        - the configuration is overwritten each time I run the sync and 
                automatic sync is always on - might be a code issue

4 System integration
        From the above consequently:
        - there should be an option to create multiple Calendars (patch exists)
        - there should be an option to separate Phonebooks 
                - I am not sure how different accounts are handled because I do
                        not keep contacts or calendar data on Google & Co
                        but I think they are separated and perhaps linked

It is not too much and it is not too little and as already mentioned it is
affecting core components, so Jolla should be involved in terms of
providing at least guidance and consulting - time is precious on both ends,
but with minimum effort we can do a great work here.

thank you in advance


More information about the Devel mailing list