Commit Graph

265 Commits

Author SHA1 Message Date
Edward Lee
5a3c53fd6a Merge weave to weave-partial-view w/ conflicts. 2009-08-31 18:03:35 -07:00
Edward Lee
f8541ddf2f Pick a canonical guid to keep in a way that all machines can agree without having all data -- "smaller" guid. 2009-08-31 17:51:26 -07:00
Edward Lee
994b05a7d1 Implement multiple ids deletion for use with 0.3 APIs. 2009-08-31 17:50:23 -07:00
Edward Lee
97e3f746cb Switch end-of-sync deletes to something SyncEngine is aware of and runs at _syncFinish. 2009-08-31 17:30:34 -07:00
Edward Lee
5468320159 Convert uses of recordLike/findLikeId to findDupe except Bookmarks is unimplemented. 2009-08-31 17:27:30 -07:00
Edward Lee
4175f5ee7e Resolve lastChannel removal and 0.5 api changes. r=thunder 2009-08-26 16:09:48 -07:00
Edward Lee
9fd47b7f98 Merge weave to weave-0.5-api w/ conflicts. 2009-08-26 15:42:05 -07:00
Edward Lee
009411f482 Bug 511746 - Resource.foo shouldn't throw except in exceptional cases. r=thunder
Get rid of lastChannel and return a String object from _request with additional properties of status, succeeded, headers -- even if the response was handled by cache. Update engines to check for non-success and throw the failure. Update tests to use these additional properties instead of lastChannel, etc.
2009-08-26 15:32:46 -07:00
Edward Lee
ca6fe1108c Backed out changeset 129ca9a54aed due to burning
test_auth_manager:      FAIL
test_resource:  FAIL
2009-08-26 01:50:36 -07:00
Anant Narayanan
371e639357 Change Resource.get() semantics and support X-Weave-Alert (bug #478330)
--HG--
extra : rebase_source : e0eb1e34f85ddd041005d780590640005dc0b434
2009-08-26 00:05:57 -07:00
Edward Lee
8431f6ff7f Read out the timestamp of the POST response header because the modified time is not part of the response data. 2009-08-25 18:04:46 -07:00
Anant Narayanan
e7097f319f Bug 507433 - Update client to use the weave 0.5 server. r-Mardak
Initial 0.5 server API switch not quite working.
2009-08-25 17:06:13 -07:00
Edward Lee
dfe8e7e513 Don't include changes to special places Library "exclude from backup" items. 2009-08-25 16:15:05 -07:00
Edward Lee
e6732d5263 Conditionally log trace records to avoid always doing record.toString().
--HG--
extra : rebase_source : 65d8025324ed95eac29d530e2699a2f29c8a2cb7
2009-08-20 17:00:15 -07:00
Edward Lee
014c01848d Print out the stack when failing to apply an incoming record. 2009-08-16 12:39:23 -07:00
Edward Lee
2335b057db Get rid of depth and sort on index instead. 2009-08-15 00:56:27 -07:00
Edward Lee
3ca750f464 Remove meta/mini records. Bye! 2009-08-13 18:50:54 -07:00
Edward Lee
6ddbaa9a8e Bug 506795 - Handle adding duplicate logins. r=thunder
Refactor reconcile to call findLikeId that by default will look for recordLike in the outgoing queue as it does now. Override findLikeId for password engine to search local logins.
2009-07-28 10:06:02 -07:00
Edward Lee
551f097468 Convert various debug messages to trace. 2009-07-22 21:40:18 -07:00
Dan Mills
db7966a67b Bug 505906: Don't do a last POST if there is nothing left to send. Regression from bug 481347. r=mardak 2009-07-22 17:48:47 -07:00
Edward Lee
a26f0b1166 Bug 485573 - Fennec runs out of memory and crashes if too many history items to sync. r=thunder
Incrementally process records as the collection finds record boundaries and converts them to records for the engine to use. Get rid of the collection iterator and original RecordParser. Add tests for incremental record parsing and remove old iter tests.
2009-07-22 16:38:34 -07:00
Dan Mills
7f9996ac73 Bug 481347: Split uploads into multiple chunks to get around server upload limit and curtail memory usage 2009-07-22 16:21:33 -07:00
Edward Lee
baa48207f2 Bug 504387 - Cache keys after creating/PUTting them instead of after GETting. r=thunder
Set the record manager for CryptoMetas, PubKeys, PrivKeys after uploading keys.
2009-07-16 23:13:58 -07:00
Edward Lee
68d361380a Bug 504230 - Use Engine's logging level for Store and Tracker
Use the log.logger.engine.* prefs to determine what the Store and Tracker should log with. This requires putting the same engine name on each Store/Tracker object, so there's some duplicate string values right now.
2009-07-14 18:34:03 -07:00
Edward Lee
e94143f4e2 Bug 504212 - Have javascript stack traces show [object Object] for various CryptoWrapper functions
Pass the ID object instead of reading out the password and passing it in to encrypt/decrypt.
2009-07-14 16:51:04 -07:00
Edward Lee
2b33ab565b Bug 504177 - Don't block the UI when syncing up/down
Sync.sleep(0) to let the main thread do UI, etc. after every record encrypt or decrypt.
2009-07-14 14:01:26 -07:00
Edward Lee
84c52a6411 merge weave. resolve addition of kSpecialIds to bookmarks.js near removal of Async.sugar. 2009-06-23 22:34:57 -07:00
Edward Lee
ad5f01ba04 Bug 493816 - Timestamps need to be saved as string prefs
Internally store .lastSync as a string but keep exposing set/getters as float values. parseFloat takes both strings and numbers and gives a number.

--HG--
extra : rebase_source : b657dd3f732b9f3b3b0ef208ee53e799aa87d2c6
2009-06-16 17:22:59 -07:00
Edward Lee
4084603da5 Remove async.js and wrap.js and remove remaining references to Async.sugar, etc. 2009-06-06 18:24:30 -05:00
Edward Lee
1b781dda8c Remove Async.sugar from engines.js. 2009-06-05 15:34:39 -07:00
Edward Lee
48b4ac18ef Engine_wipeServer: async -> sync. SyncEngine__wipeServer: async/yield -> sync. 2009-06-05 15:34:37 -07:00
Edward Lee
f9aeec8b2c Engine_wipeClient: async -> sync. Engine__wipeClient: async/yield -> sync. 2009-06-05 15:34:35 -07:00
Edward Lee
3d3f786723 Engine_resetClient: async -> sync. SyncEngine__resetClient: async/yield -> sync. 2009-06-05 15:34:34 -07:00
Edward Lee
81f7652b56 Engine_sync: async -> sync. SyncEngine__sync: async/yield -> sync. 2009-06-05 15:34:32 -07:00
Edward Lee
4f22e547b2 Temporarily convert _notify to _notifyAsync and add a Utils.notify. 2009-06-05 15:34:23 -07:00
Edward Lee
563c0a0fd0 SyncEngine__processIncoming: async/yield -> sync. 2009-06-05 00:39:35 -07:00
Edward Lee
e7a503025a CollectionIterator_next: async + async/yield -> sync. 2009-06-05 00:38:23 -07:00
Edward Lee
3a4f2ca0ee Store_applyIncoming, SyncEngine__applyIncoming: async + async/yield -> sync. 2009-06-05 00:36:11 -07:00
Edward Lee
fce5aa36de SyncEngine__syncStartup, SyncEngine__uploadOutgoing, SyncEngine__syncFinish all async/yield -> sync. 2009-06-04 23:48:27 -07:00
Edward Lee
228f2e2568 CryptoWrapper_decrypt: async + async/yield -> sync. Fix up bookmark/client decrypt. 2009-06-04 19:06:57 -07:00
Edward Lee
7adf2ffa78 CryptoWrapper_encrypt: async + async/yield -> sync. 2009-06-04 17:58:37 -07:00
Edward Lee
a9de56fef0 CryptoMeta_addUnwrappedKey: async + async/yield -> sync. 2009-06-04 17:36:34 -07:00
Edward Lee
df16a63edb PubKeyManager_getDefaultKey: async + async/yield -> sync. 2009-06-04 17:04:51 -07:00
Edward Lee
ce37486d9c RecordMgr_get: async + async/yield -> sync. 2009-06-04 16:50:57 -07:00
Edward Lee
68ce1e4224 Change SyncEngine._reconcile to not be async/yield. 2009-06-04 14:18:04 -07:00
Edward Lee
1134960b32 Fix Resource.* call sites used for wiping the server and initial sync (+ key gen upload) 2009-06-04 10:17:27 -07:00
Edward Lee
428d54a14a Change Resource.get() to be sync (no callback) and fix up call sites used for login + sync now. 2009-06-04 10:16:07 -07:00
Edward Lee
232befb843 Bug 493256 - Reconcile doesn't handle removed items correctly. r=thunder
Have engines check if the deleted flag is the same for _isEqual.
2009-05-15 13:18:16 -05:00
Edward Lee
665e488e28 Bug 493001 - Dynamically generate weave prefs engine list based on registered engines
Remove existing hardcoded engine list + prefs and build checkboxes based on registered engines. Engines that give null for enabled (instead of true/false) are considered disabled. Update base Engine enabled get/set to give null and allow setting.
2009-05-14 11:38:15 -05:00
Dan Mills
76438f3876 skip getting meta records for records already in the meta list 2009-04-08 18:08:45 -07:00
Edward Lee
a7b212ab0b Bug 486810 - Engines (and their trackers) are created with every new window. r=thunder
Register the built-in engines on service start-up instead of from the overlay, and have Engines.register check if the engine has already been registered.
2009-04-08 14:39:14 -05:00
Edward Lee
75bacf714f Bug 487308 - Allow registering of an array of engines
Import engines to the Weave global object and use them to register engines, which checks if the arg is an array. To support handling of errors (unused), the engine is returned on register failure.
2009-04-07 16:45:41 -05:00
Edward Lee
cebb90b5bc Add a Utils.isArray and use it 2009-04-07 16:45:29 -05:00
Edward Lee
3993035a59 Bug 482670 - WBORecord should implement a .deleted property
Expose a .deleted property that engines can set to true to store an even thinner deleted payload (empty string "" instead of "[null]") on the server. Handle deserializing of deleted records by setting the property. Note: Engines must set their payloads to something JSON-able if it's not a delete record.
2009-04-03 12:38:47 -05:00
Edward Lee
4f6ebe6be5 Bug 481741 - Switch to JSON from nsIJSON 2009-03-31 14:09:38 -05:00
Edward Lee
75994bc68f Bug 482898 - Allow an optional array of engines for wipeRemote, wipeServer, resetClient, etc.. r=thunder 2009-03-20 00:13:16 -07:00
Edward Lee
e2f30efcf4 self.cb not this.cb for engine._wipeClient 2009-03-11 14:35:23 -05:00
Dan Mills
7b9eedde66 Bug 478327: Send minimal '[null]' payload for deleted records 2009-03-11 01:40:04 -07:00
Edward Lee
edda1db597 Bug 480457 - Expose a resetClient API for weave service and engines. r=thunder 2009-02-26 22:36:14 -08:00
Dan Mills
01ab4aa284 add some trace-level debugging to reconciler; ignore old (changed) IDs while reconciling 2009-03-03 01:29:35 -08:00
Dan Mills
3e1235a9e5 Switch away from __defineGetter__ due to bug 481104; explicitly cache logins in password engine; whitespace fixes 2009-03-02 18:55:26 -08:00
Anant Narayanan
0f6fdde06c Remove getter that fixes the 'two-store' problem 2009-03-03 01:15:48 +01:00
Dan Mills
bfa22eed1a Bug 480461: skip items that cannot be decrypted 2009-02-26 17:31:06 -08:00
Dan Mills
50621a1889 remove unused observer service getter 2009-02-26 13:52:12 -08:00
Dan Mills
edcaabd226 reset lastSync pref before setting it, as some clients have mismatched types saved 2009-02-23 19:33:40 -08:00
Dan Mills
8c671b69c9 Wrap.notify now takes the topic prefix as an argument, instead of requiring this._osPrefix to be set; use Observers.js in several places (sync.js, status.js, etc); some event topics have changed, beware; fix up status window, now prints some useful status as sync progresses 2009-02-22 00:04:58 -08:00
Dan Mills
1490d0c250 shorten engine logging pref names, set defaults for them 2009-02-20 01:51:20 -08:00
Dan Mills
d0e9169992 pass the cryptoMetaURL to the store's createRecord 2009-02-19 00:36:55 -08:00
Dan Mills
1883c83218 make sure to only save the last sync timestamp as a number 2009-02-18 20:17:30 -08:00
Dan Mills
a6a7141691 add a method for resetting the engine's lastSync pref 2009-02-17 13:20:02 -08:00
Dan Mills
a30d82e53d add '-engine' to engine name in observer notifications 2009-02-11 21:51:24 -08:00
Dan Mills
57c5368d7a remove sm-weave-proxy01 hack 2009-02-11 13:47:51 -08:00
Dan Mills
aafa72814f multi-cluster support; use new 'newer' flag instead of 'modified' for collection searches 2009-02-10 00:57:16 -08:00
Dan Mills
03d78b3d84 set record IDs inside the store; fix bookmarks tracker, it was ignoring change events (e.g., changed title and such) 2009-01-27 16:54:54 -08:00
Dan Mills
a47fcda5b1 only url-encode weave IDs when constructing a URL with them 2009-01-27 16:36:00 -08:00
Dan Mills
a3d9640e3c Make records not extend Resource, instead they have a serialize() method that can be used to create a Resource for uploading if necessary.
Use global service instances under Svc.*
Consolidate various record managers (caches/convenience factories) to extend the same base object
Log exceptions caught by notify wrapper
2009-01-27 13:35:10 -08:00
Dan Mills
84340dcdc2 cast modified timestamp to an int if it's a string 2009-01-21 19:02:38 -08:00
jonathandicarlo@jonathan-dicarlos-macbook-pro.local
e823363952 Merged 2009-01-15 14:06:00 -08:00
jonathandicarlo@jonathan-dicarlos-macbook-pro.local
cd797f05a7 Added a method to historyStore called tempTableExists which does a SELECT (rather than using tableExists) to correctly tell whether a temp table is there or not, so that historyEngine will work correctly on both Fennec and Firefox (See bug 472963) 2009-01-15 14:05:50 -08:00
Dan Mills
67f99f74cd add some extra log info to engine; have tracker ignore all changes when applying a change to prevent the tracker from generating new guids for new items before the store has a chance to set the right one 2009-01-14 22:01:04 -08:00
Dan Mills
445916b2e6 ignore/unignore specific weave IDs instead of a blanket enable/disable of the tracker during sync. 2009-01-13 14:43:21 -08:00
Dan Mills
cf5c78dfcf skip part of reconciliation for incoming deleted records 2009-01-08 21:33:37 -08:00
Dan Mills
c3dc0195a6 fail recordLike if either record is null (deleted) 2009-01-07 13:05:05 -08:00
Dan Mills
ff240dd3ce add clients sync (list of clients with names and types); make it simpler to create Engine subclasses; remove the 2nd 'authenticator' parameter for Resources; resetServer is now wipeServer; consider 0 an invalid place ID (fixes bug when calculating item depths); add a Svc global exported from utils.js where commonly used services can live 2009-01-06 13:54:18 -08:00
Dan Mills
34a114606d fix depth/index record generation 2009-01-02 21:13:32 -08:00
Dan Mills
cc914050b2 drop _isEqual debug statements to trace level 2009-01-02 17:35:47 -08:00
Dan Mills
41bd8b02b9 consider sortindex when checking if a remote record is the same as a local one 2009-01-02 17:27:45 -08:00
Dan Mills
4cb774d408 remove extra debug code 2009-01-02 16:17:36 -08:00
Dan Mills
2754039971 fix/further simplify reconciliation 2009-01-02 15:51:35 -08:00
Dan Mills
62c5aa360c remove 'outgoing' record cache (generate outgoing records at send time); clear the store cache and force a GC after reconciliation; push depth+index (metadata) records after regular (full) records; simplify + better comment reconciliation function; fix low mem check 2009-01-02 13:51:38 -08:00
Dan Mills
d04e52f8f5 typo fix 2008-12-30 23:56:53 -08:00
Dan Mills
5e3ac9586e add a generic cache class for storing records during reconciliation. cache has 100 item limit, and is cleared before and after reconciliation so the OS can reclaim memory after GC 2008-12-30 23:52:20 -08:00
Dan Mills
380b614133 make engine keep less records in memory by limiting the outgoing queue to a maximum of 100 records, and fetch the rest from the store each time 2008-12-29 23:28:17 -08:00
Dan Mills
6b260498e0 switch to generating individual bookmark records directly from the places store, without using a cache (to improve memory performance); create objects for bookmark types; temporarily disable history sync 2008-12-28 19:59:44 -08:00
Dan Mills
8a57024bf8 force GC, makes sync of many items possible on fennec 2008-12-26 21:49:17 -08:00
Dan Mills
aff056726e change serverURL to use 0.3 url schema 2008-12-23 13:51:30 -08:00
Dan Mills
62a45ad0ab Various engine/bookmark changes:
* Rely on the server to sort incoming records, remove all sorting code client-side.
* Streamline sync to be able to process incoming records one at a time, as soon as they are downloaded.  This changes reconciliation to be able to process a single incoming record.
* Engine base class will automatically convert parentid, depth, and sortindex into toplevel WBO objects.  This is good for now but kinda broken, engine subclasses should really be generating WBO records themselves.
* Since index is now a toplevel WBO property, there is no need for the bookmarks sync code to subclass recordLike.
* Refactor bookmarks store to be a little cleaner and work directly with records instead of "commands".
2008-12-23 11:30:31 -08:00
Dan Mills
08512b99a5 change engine to process (download, reconcile, apply changes) incoming records one by one 2008-12-19 11:48:09 -08:00
Dan Mills
d55df6a1f4 implement the awe (or tears) inducing places-style sql queries -- history sync works pretty well now 2008-12-18 22:39:32 -08:00
Dan Mills
1e7370ba2d don't stop applying changes when one fails to do so 2008-12-12 13:53:48 -08:00