[copying meego-dev, as originally intended]
Hello Buteo team!
I'd like to share the result of Buteo SyncML interoperability testing
between MeeGo and Google Contacts. Credits go to Yongsheng and Qiankun
for doing the actual work on this. I'll also send similar emails for
some other areas soon.
The goal of this work was:
* two-way synchronization with one peer (Google),
* without data loss,
* with as much data getting synchronized as possible.
Among the peers which are meant to be supported [1], we picked the one
which was seen as the most important one. As described in [2], data loss
occurs when importing updated contacts back from Google into QtContacts,
both because not all data was sent to Google (QContactOnlineAccount) and
because Google doesn't store all data sent to it.
My understanding is that the Buteo components (SyncML plugin, storage
plugins) are reference implementations. They work at a basic level, but
any kind of customization has to be done in the context of specific
products, if necessary by modifying the reference plugins. Following
that approach, we patched the hcontacts plugin to make it work well with
Google.
But because in this case meego.com is the product, and there is the
requirement to support Google in it, this patch (or a suitable
replacement) should be included in the core MeeGo package
buteo-sync-plugins. I'm not sure what the best approach for that is
though, please advice.
The options that I see are:
1. Add a hcontacts-google plugin with its own source code.
2. Maintain one hcontacts plugin, with peer specific code inside
if() branches.
A. The condition could be a XML config option or
B. auto-detection at runtime based on DevInf information
sent by the peer.
Of these options, 2.A seems to be the easiest to me. We haven't added it
yet because a) we wanted to get feedback first and b) the current XML
format was said to be preliminary, so modifying it might have conflicted
with changes currently being made at Nokia.
I'm attaching the code changes as patches and also pushed them into the
"google" branch of [3]. If you prefer merge requests over "git am", then
I can create such merge requests. The patches apply to the current
master.
Most of the patches have a commit message and comments that explain
them. Let me just add something missing from the description of the
largest of these patches, the one that does a partial update of a local
contact: the code was inspired and partially copied under LGPL from the
code added to QtContacts [4]. It turned out that we couldn't use the new
QtContacts API because it didn't give enough control over updating. With
the API it is only possible to choose between "overwrite
QtContactOrganization" or "keep it", but Google supports a subset of
that detail.
[1] http://bugs.meego.com/show_bug.cgi?id=5468 "[FEA] SyncUI - Destinations"
[2] http://bugs.meego.com/show_bug.cgi?id=5607 "[FEA] two-way sync without data
loss + sync as much data as possible"
[3] http://meego.gitorious.org/~pohly/meego-middleware/pohlys-buteo-syncfw
[4] http://bugs.meego.com/show_bug.cgi?id=4897#c28 "contacts: two-way sync with
less capable peer without data loss"
--
Best Regards
Patrick Ohly
Senior Software Engineer
Intel GmbH
Open Source Technology Center
Pützstr. 5 Phone: +49-228-2493652
53129 Bonn
Germany
>From d44ab17d7bad5eef8d1af75ea1ca2f47ac2e2019 Mon Sep 17 00:00:00 2001
From: Zhu, Yongsheng <[email protected]>
Date: Tue, 30 Nov 2010 03:38:13 -0500
Subject: [PATCH 1/4] Google contact: only update fields Google supports
We only update those fields Google supports. Otherwise,
just keep them unchanged to avoid data loss.
The temporary solution is to use 'partial save' of qtcontacts.
We never update fields Google never touches when doing sync with it.
Copy code from http://qt.gitorious.org/qt-mobility/contacts/
blobs/master/src/contacts/qcontactmanagerengine.cpp under LGPLv2.1.
---
storageplugins/hcontacts/ContactsBackend.cpp | 185 +++++++++++++++++++++++++-
storageplugins/hcontacts/ContactsBackend.h | 4 +
2 files changed, 187 insertions(+), 2 deletions(-)
diff --git a/storageplugins/hcontacts/ContactsBackend.cpp b/storageplugins/hcontacts/ContactsBackend.cpp
index fdc76f5..df2cb10 100644
--- a/storageplugins/hcontacts/ContactsBackend.cpp
+++ b/storageplugins/hcontacts/ContactsBackend.cpp
@@ -30,6 +30,18 @@
#include <QContactLocalIdFilter>
#include <QContactSyncTarget>
#include <QContactDetailFilter>
+#include <QContactAddress>
+#include <QContactBirthday>
+#include <QContactEmailAddress>
+#include <QContactDisplayLabel>
+#include <QContactName>
+#include <QContactNote>
+#include <QContactOrganization>
+#include <QContactPhoneNumber>
+#include <QContactThumbnail>
+#include <QContactUrl>
+#include <QContactGender>
+#include <QContactOnlineAccount>
#include <QBuffer>
#include <QSet>
@@ -167,7 +179,7 @@ bool ContactsBackend::addContacts( const QStringList& aContactDataList,
i->saveDetail(&syncTarget);
}
- bool retVal = iMgr->saveContacts(&contactList, &errorMap);
+ bool retVal = saveContacts(&contactList, &errorMap);
if (!retVal)
{
@@ -265,7 +277,7 @@ QMap<int,ContactsStatus> ContactsBackend::modifyContacts(
LOG_DEBUG("Replacing item's ID " << qContactList.at(i).localId());
}
- if(iMgr->saveContacts(&qContactList , &errors)) {
+ if(saveContacts(&qContactList , &errors)) {
LOG_DEBUG("Batch Modification of Contacts Succeeded");
}
else {
@@ -663,3 +675,172 @@ QContactFilter ContactsBackend::getSyncTargetFilter() const {
// return the union
return detailFilterButeoSyncTarget | detailFilterDefaultSyncTarget;
}
+
+bool ContactsBackend::saveContacts( QList<QContact> *contactList,
+ QMap<int, QContactManager::Error> *errorMap )
+{
+ struct DefinitionFieldPair {
+ QString definitionName;
+ QString fieldName;
+ };
+
+ static DefinitionFieldPair googleFields[] = {
+ {QContactAddress::DefinitionName, ""}, // ADR
+ {QContactBirthday::DefinitionName, QContactBirthday::FieldBirthday}, // BDAY
+ {QContactEmailAddress::DefinitionName, QContactEmailAddress::FieldEmailAddress}, // EMAIL
+ {QContactDisplayLabel::DefinitionName, ""}, //FN
+ {QContactName::DefinitionName, ""}, //N
+
+ {QContactNote::DefinitionName, QContactNote::FieldNote}, //NOTE
+ {QContactTimestamp::DefinitionName, ""}, //REV
+ {QContactPhoneNumber::DefinitionName, QContactPhoneNumber::FieldNumber}, //TEL
+ {QContactPhoneNumber::DefinitionName, QContactPhoneNumber::SubTypeAssistant}, //X-ASSISTANT-TEL
+ {QContactThumbnail::DefinitionName, ""}, //PHOTO
+
+ {QContactUrl::DefinitionName, QContactUrl::FieldUrl}, //URL
+ {QContactGender::DefinitionName, QContactGender::FieldGender}, //X-GENDER
+ {QContactOnlineAccount::DefinitionName, QContactOnlineAccount::SubTypeSip}, //X-SIP
+
+ // separator
+ {"", ""},
+
+ // below are stored in one detail in contact, different from above ones
+ {QContactOrganization::DefinitionName, QContactOrganization::FieldName}, //ORG
+ {QContactOrganization::DefinitionName, QContactOrganization::FieldTitle}, //TITLE
+ {QContactOrganization::DefinitionName, QContactOrganization::FieldAssistantName} //X-ASSITANT
+ };
+
+ // Partial contact save.
+ // Basically
+
+ // Need to:
+ // 1) fetch existing contacts
+ // 2) strip out fields of details in definitionMask for existing contacts
+ // 3) copy the details from the passed in list for existing contacts
+ // 4) for any new contacts, copy the masked details to a blank contact
+ // 5) save the modified ones
+ // 6) update the id of any new contacts
+ // 7) transfer any errors from saving to errorMap
+
+ QList<QContactLocalId> existingContactIds;
+
+ // Error conditions:
+ // 1) bad id passed in (can't fetch)
+ // 2) bad fetch (can't save partial update)
+ // 3) bad save error
+ // all of which needs to be returned in the error map
+
+ QHash<int, int> existingIdMap; // contacts index to existingContacts index
+
+ // Try to figure out which of our arguments are new contacts
+ QList<QContactLocalId> localIds = iMgr->contactIds();
+ for(int i = 0; i < contactList->count(); i++) {
+ // don't check manager uri here for they can't match
+ // See if there's a contactId that's not from this manager
+ const QContact c = contactList->at(i);
+ if ( c.localId() != 0 && localIds.indexOf(c.localId()) >= 0 ) {
+ existingIdMap.insert(i, existingContactIds.count());
+ existingContactIds.append(c.localId());
+ }
+ }
+
+ // Now fetch the existing contacts
+ QList<QContact> existingContacts;
+ getContacts(existingContactIds, existingContacts);
+
+ // Prepare the list to save
+ QList<QContact> contactsToSave;
+ QList<int> savedToOriginalMap; // contactsToSave index to contacts index
+
+ for (int i = 0; i < contactList->count(); i++) {
+ // See if this is an existing contact or a new one
+ const int fetchedIdx = existingIdMap.value(i, -1);
+ QContact contactToSave;
+ const QContact& c = contactList->at(i);
+
+ bool removeField = false;
+ unsigned int j = 0;
+ if (fetchedIdx >= 0) {
+ // Existing contact we should have fetched
+ contactToSave = existingContacts.at(fetchedIdx);
+
+ // remove existing details with definitions
+ for ( ; j < sizeof(googleFields) / sizeof(googleFields[0]); j++) {
+ if ( googleFields[j].definitionName.isEmpty() ) {
+ removeField = true;
+ continue;
+ }
+ QList<QContactDetail> details = contactToSave.details(googleFields[j].definitionName);
+ foreach(QContactDetail detail, details) {
+ if ( removeField ) {
+ // only remove related fields otherwise other fields might be removed
+ // for example, 'ROLE', 'ORG', 'TITLE', and 'X-ASSITANT' are stored in
+ // one detail. If removing the detail, 'ROLE' will be lost.
+ contactToSave.removeDetail(&detail);
+ detail.removeValue(googleFields[j].fieldName);
+ contactToSave.saveDetail(&detail);
+ } else if ( googleFields[j].fieldName.isEmpty()
+ || detail.hasValue(googleFields[j].fieldName) ) {
+ // only remove specific field instead of a total detail. Because many fields
+ // are merged into one detail
+ contactToSave.removeDetail(&detail);
+ }
+ }
+ }
+ }
+
+ // Now copy in the details from the arguments
+
+ // Perhaps this could do this directly rather than through saveDetail
+ // but that would duplicate the checks for display label etc
+ //foreach (const QString& name, definitions) {
+ removeField = false;
+ for (j = 0; j < sizeof(googleFields) / sizeof(googleFields[0]); j++) {
+ if ( googleFields[j].definitionName.isEmpty() ) {
+ removeField = true;
+ continue;
+ }
+ QList<QContactDetail> details = c.details(googleFields[j].definitionName);
+ foreach(QContactDetail detail, details) {
+ if ( removeField ) {
+ QList<QContactDetail> detailsSave = contactToSave.details(googleFields[j].definitionName);
+ if ( detailsSave.isEmpty() ) {
+ contactToSave.saveDetail(&detail);
+ } else {
+ QContactDetail detailSave = detailsSave.at(0);
+ detailSave.setValue(googleFields[j].fieldName, detail.value(googleFields[j].fieldName) );
+ contactToSave.saveDetail(&detailSave);
+ }
+ } else if ( googleFields[j].fieldName.isEmpty() ||
+ detail.hasValue(googleFields[j].fieldName) ) {
+ contactToSave.saveDetail(&detail);
+ }
+ }
+ }
+
+ savedToOriginalMap.append(i);
+ contactsToSave.append(contactToSave);
+ }
+
+ // Now save them
+ QMap<int, QContactManager::Error> saveErrors;
+ iMgr->saveContacts(&contactsToSave, &saveErrors);
+
+ // Now update the passed in arguments, where necessary
+
+ // Update IDs of the contacts list
+ for (int i = 0; i < contactsToSave.count(); i++) {
+ (*contactList)[savedToOriginalMap[i]].setId(contactsToSave[i].id());
+ }
+ // Populate the errorMap with the errorMap of the attempted save
+ QMap<int, QContactManager::Error>::iterator it(saveErrors.begin());
+ while (it != saveErrors.end()) {
+ if (it.value() != QContactManager::NoError) {
+ errorMap->insert(savedToOriginalMap[it.key()], it.value());
+ }
+ it++;
+ }
+
+ return errorMap->isEmpty();
+}
+
diff --git a/storageplugins/hcontacts/ContactsBackend.h b/storageplugins/hcontacts/ContactsBackend.h
index ce3b34f..a0fb59d 100644
--- a/storageplugins/hcontacts/ContactsBackend.h
+++ b/storageplugins/hcontacts/ContactsBackend.h
@@ -224,6 +224,10 @@ private: // functions
*/
QContactFilter getSyncTargetFilter() const;
+ // temporary solution to save contacts, update specific fields
+ bool saveContacts( QList<QContact> *contactList,
+ QMap<int, QContactManager::Error> *errorMap );
+
private: // data
// if there is more than one Manager we need to have a list of Managers
--
1.7.2.3
>From 38bc658f05fbb8dbffd22c5e6e457cca435ba411 Mon Sep 17 00:00:00 2001
From: Qiankun Miao <[email protected]>
Date: Fri, 3 Dec 2010 02:55:58 +0800
Subject: [PATCH 2/4] contactbackend: add PHOTO support
When doing sync, photo of a contact would be lost due to contactbackend
doesn't handle PHOTO property. This patch fixes this bug (BMC#5879)
PHOTO information of a contact is stored as a picture in
"~/.cache/contacts/photo/".
And the path of the picture can be get from QContactAvatar detail. Read
content of the picture from this path, then write as QContactThumbnail
detail, finally PHOTO property would appear in VCard format of the
contact.
---
storageplugins/hcontacts/ContactsBackend.cpp | 14 +++++++++++++-
1 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/storageplugins/hcontacts/ContactsBackend.cpp b/storageplugins/hcontacts/ContactsBackend.cpp
index df2cb10..a6dcbda 100644
--- a/storageplugins/hcontacts/ContactsBackend.cpp
+++ b/storageplugins/hcontacts/ContactsBackend.cpp
@@ -44,6 +44,8 @@
#include <QContactOnlineAccount>
#include <QBuffer>
#include <QSet>
+#include <QContactThumbnail>
+#include <QContactAvatar>
const QLatin1String ButeoSyncTarget("buteo");
@@ -422,8 +424,18 @@ QString ContactsBackend::convertQContactToVCard(const QContact &aContact)
{
FUNCTION_CALL_TRACE;
+ QContact tContact = aContact;
+ const QContactAvatar avatar = tContact.detail(QContactAvatar::DefinitionName);
+ const QContactThumbnail thumb = tContact.detail(QContactThumbnail::DefinitionName);
+ if (!avatar.isEmpty() && thumb.isEmpty()){
+ QImage image(avatar.imageUrl().path());
+ QContactThumbnail thumbnail;
+ thumbnail.setThumbnail(image);
+ tContact.saveDetail(&thumbnail);
+ }
+
QList<QContact> contactsList;
- contactsList.append (aContact);
+ contactsList.append (tContact);
QVersitContactExporter contactExporter;
--
1.7.2.3
>From 6221b4b571e6a2b8caa58e9a817f09dee4e13204 Mon Sep 17 00:00:00 2001
From: Zhu, Yongsheng <[email protected]>
Date: Thu, 13 Jan 2011 11:17:02 +0100
Subject: [PATCH 3/4] Google service profile: removed redundant username/password
Credentials are configured inside the sync profile, not the service
profile. At best they are ignored in the sync profile, in the worst
case they confuse people reading the XML files. Therefore this patch
removes them.
---
.../syncmlclient/xml/service/google.com.xml | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
diff --git a/clientplugins/syncmlclient/xml/service/google.com.xml b/clientplugins/syncmlclient/xml/service/google.com.xml
index 595efa9..2066a80 100644
--- a/clientplugins/syncmlclient/xml/service/google.com.xml
+++ b/clientplugins/syncmlclient/xml/service/google.com.xml
@@ -3,9 +3,6 @@
<profile name="google.com" type="service">
<key name="Remote database" value="https://m.google.com/syncml"; />
- <key name="Username" value="enter username" />
- <key name="Password" value="enter passwd" />
-
<key name="destinationtype" value="online"/>
<profile name="syncml" type="client" >
--
1.7.2.3
>From 170d2525d1538b5856b89ea6feb3a0ee97ac32d1 Mon Sep 17 00:00:00 2001
From: Zhu, Yongsheng <[email protected]>
Date: Thu, 13 Jan 2011 11:18:58 +0100
Subject: [PATCH 4/4] Google sync profile: added essential "syncml" type
Synchronization fails to start unless the SyncML client plugin
is specified explicitly. This patch adds that.
---
clientplugins/syncmlclient/xml/sync/google.com.xml | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/clientplugins/syncmlclient/xml/sync/google.com.xml b/clientplugins/syncmlclient/xml/sync/google.com.xml
index fa280bb..b8dc0e9 100644
--- a/clientplugins/syncmlclient/xml/sync/google.com.xml
+++ b/clientplugins/syncmlclient/xml/sync/google.com.xml
@@ -7,6 +7,8 @@
<profile type="service" name="google.com" >
</profile>
+ <profile name="syncml" type="client" >
+ </profile>
<profile name="hcontacts" type="storage" >
<key name="enabled" value="true" />
--
1.7.2.3
_______________________________________________
MeeGo-dev mailing list
[email protected]
http://lists.meego.com/listinfo/meego-dev
