Commit Graph

288 Commits

Author SHA1 Message Date
Edward Lee
13cddbf9bc Bug 520215 - Only wipe if data is able to be restored from the server
For each engine, try decrypting a record from the server before wiping local data.
2009-10-12 16:22:54 -07:00
Edward Lee
a994e4258f Simplify logic for Engines.getAll/Enabled and use Enabled for main sync loop. 2009-10-08 13:51:22 -07:00
Edward Lee
32f373dcc0 Don't unconditionally forceGC on processIncoming especially when there's nothing to process. 2009-09-25 18:39:24 -07:00
Edward Lee
a69c3b332a Sort the sync timing report to show certain functions first. 2009-09-25 18:38:47 -07:00
Edward Lee
4f389c5c8a Bug 518972 - Only upload history records that have more than one visit
Allow each engine to provide a custom Collection object and have History provide a collection that filters out certain data. This is inefficient because we have to first create then encrypt the record before we can filter it out.
2009-09-25 16:52:12 -07:00
Edward Lee
eba12cbe8d Bug 518018 - Default to serverURL instead of userAPI on 404 cluster check
Rework server/user/misc prefs to allow relative paths and full urls for generating API paths. Cache string properties of generated URLs under the storageAPI instead of using dynamic getters.
2009-09-21 17:13:41 -07:00
Edward Lee
8db1de49fa Merge weave to about-weave. 2009-09-16 19:16:48 -07:00
Edward Lee
b48bdeef35 Remove meta serialization now that resource takes objects. 2009-09-16 19:15:55 -07:00
Dan Mills
5094a04f3d initial patch
--HG--
extra : rebase_source : 250c8f0d85483300147a6f0d691e0214db6d837b
2009-09-15 21:54:05 -04:00
Mike Connor
4faaba2018 bug 511549 - make detailedStatus much smarter about errors and backoff, r=edilee
--HG--
extra : rebase_source : 42e3eb9c89d2567d18c02e7583202e284dd69b0e
2009-09-15 21:38:52 -04:00
Edward Lee
91e977a052 Fix exception name. 2009-09-11 13:10:19 -07:00
Edward Lee
e856402be7 Add some comments to partial download code and remove other comments. 2009-09-11 08:24:42 -07:00
Edward Lee
9380981a9e Merge weave to weave-partial-view. 2009-09-10 23:18:33 -07:00
Edward Lee
fb1bc5da49 Bug 507429 - Partial sync download support
Only fetch a limited number of items on first/update syncs and if we get the same number, ask the server for the ids to fetch later. Also on every download, process some of the backlog and save the list of GUIDs to disk as json for cross-session support.
2009-09-10 23:11:33 -07:00
Edward Lee
8b8447ab6e Remove some unnecessary logs that follow right after the notify/event now that we only show verbose. Only log non-success response body for Trace. 2009-09-10 22:57:36 -07:00
Edward Lee
b59b775794 Merge weave to weave-partial-view. 2009-09-10 12:42:29 -07:00
Edward Lee
8f5372e6b1 Update lastSync to the lastModified time from info/collections instead of updating times on each record. This is to prepare for fetching a list of GUIDs and processing a list of GUIDs. 2009-09-10 12:41:38 -07:00
Edward Lee
b0c3b9005e Bug 515676 - Allow giving objects to PUT/POST without Filters to avoid stringify([parse(stringify(stringify(obj)))])
Get rid of Filters and automatically JSON.stringify PUT/POST data that aren't strings, so plain Records can be passed in to PUT and POST. This leverages toJSON of Records to provide an object that can be serialized. Fix up client record serialize/deserialize to still escape/unescape non-ASCII.
2009-09-10 11:05:13 -07:00
Edward Lee
0dc7cf0d81 Merge weave to weave-partial-view. 2009-09-08 23:35:21 -07:00
Edward Lee
258a985cee Expose a _handleDupe on engines and provide a custom one for bookmarks that tracks GUID changes so that it can keep an alias mapping to fix incoming item properties (id, parent, predecessor). Move out _reparentOrphans so that it is triggered on update and not just create because folders can change ids to the right parent. 2009-09-08 23:33:15 -07:00
Edward Lee
19443b737b Bug 514323 - Use info/collections to check if there's new data. r=thunder
Fetch info/collections before syncing engines and set the lastModified property on the engine so that they can check if they need to process incoming.
2009-09-03 21:30:40 -07:00
Edward Lee
9ec91eed07 Bug 514600 - Track how long various parts of sync take
Instrument all functions that are part of the sync engine (except some constructors, etc.) and generate statistics (min/max/sum/num/avg) for processing. For now with the default appender, implement toString to report just the total time.
2009-09-03 20:11:36 -07:00
Edward Lee
0fa22bedac Switch to 0.5 server API for deleting multiple ids from a collection. 2009-08-31 18:30:44 -07:00
Edward Lee
5a3c53fd6a Merge weave to weave-partial-view w/ conflicts. 2009-08-31 18:03:35 -07:00
Edward Lee
f8541ddf2f Pick a canonical guid to keep in a way that all machines can agree without having all data -- "smaller" guid. 2009-08-31 17:51:26 -07:00
Edward Lee
994b05a7d1 Implement multiple ids deletion for use with 0.3 APIs. 2009-08-31 17:50:23 -07:00
Edward Lee
97e3f746cb Switch end-of-sync deletes to something SyncEngine is aware of and runs at _syncFinish. 2009-08-31 17:30:34 -07:00
Edward Lee
5468320159 Convert uses of recordLike/findLikeId to findDupe except Bookmarks is unimplemented. 2009-08-31 17:27:30 -07:00
Edward Lee
4175f5ee7e Resolve lastChannel removal and 0.5 api changes. r=thunder 2009-08-26 16:09:48 -07:00
Edward Lee
9fd47b7f98 Merge weave to weave-0.5-api w/ conflicts. 2009-08-26 15:42:05 -07:00
Edward Lee
009411f482 Bug 511746 - Resource.foo shouldn't throw except in exceptional cases. r=thunder
Get rid of lastChannel and return a String object from _request with additional properties of status, succeeded, headers -- even if the response was handled by cache. Update engines to check for non-success and throw the failure. Update tests to use these additional properties instead of lastChannel, etc.
2009-08-26 15:32:46 -07:00
Edward Lee
ca6fe1108c Backed out changeset 129ca9a54aed due to burning
test_auth_manager:      FAIL
test_resource:  FAIL
2009-08-26 01:50:36 -07:00
Anant Narayanan
371e639357 Change Resource.get() semantics and support X-Weave-Alert (bug #478330)
--HG--
extra : rebase_source : e0eb1e34f85ddd041005d780590640005dc0b434
2009-08-26 00:05:57 -07:00
Edward Lee
8431f6ff7f Read out the timestamp of the POST response header because the modified time is not part of the response data. 2009-08-25 18:04:46 -07:00
Anant Narayanan
e7097f319f Bug 507433 - Update client to use the weave 0.5 server. r-Mardak
Initial 0.5 server API switch not quite working.
2009-08-25 17:06:13 -07:00
Edward Lee
dfe8e7e513 Don't include changes to special places Library "exclude from backup" items. 2009-08-25 16:15:05 -07:00
Edward Lee
e6732d5263 Conditionally log trace records to avoid always doing record.toString().
--HG--
extra : rebase_source : 65d8025324ed95eac29d530e2699a2f29c8a2cb7
2009-08-20 17:00:15 -07:00
Edward Lee
014c01848d Print out the stack when failing to apply an incoming record. 2009-08-16 12:39:23 -07:00
Edward Lee
2335b057db Get rid of depth and sort on index instead. 2009-08-15 00:56:27 -07:00
Edward Lee
3ca750f464 Remove meta/mini records. Bye! 2009-08-13 18:50:54 -07:00
Edward Lee
6ddbaa9a8e Bug 506795 - Handle adding duplicate logins. r=thunder
Refactor reconcile to call findLikeId that by default will look for recordLike in the outgoing queue as it does now. Override findLikeId for password engine to search local logins.
2009-07-28 10:06:02 -07:00
Edward Lee
551f097468 Convert various debug messages to trace. 2009-07-22 21:40:18 -07:00
Dan Mills
db7966a67b Bug 505906: Don't do a last POST if there is nothing left to send. Regression from bug 481347. r=mardak 2009-07-22 17:48:47 -07:00
Edward Lee
a26f0b1166 Bug 485573 - Fennec runs out of memory and crashes if too many history items to sync. r=thunder
Incrementally process records as the collection finds record boundaries and converts them to records for the engine to use. Get rid of the collection iterator and original RecordParser. Add tests for incremental record parsing and remove old iter tests.
2009-07-22 16:38:34 -07:00
Dan Mills
7f9996ac73 Bug 481347: Split uploads into multiple chunks to get around server upload limit and curtail memory usage 2009-07-22 16:21:33 -07:00
Edward Lee
baa48207f2 Bug 504387 - Cache keys after creating/PUTting them instead of after GETting. r=thunder
Set the record manager for CryptoMetas, PubKeys, PrivKeys after uploading keys.
2009-07-16 23:13:58 -07:00
Edward Lee
68d361380a Bug 504230 - Use Engine's logging level for Store and Tracker
Use the log.logger.engine.* prefs to determine what the Store and Tracker should log with. This requires putting the same engine name on each Store/Tracker object, so there's some duplicate string values right now.
2009-07-14 18:34:03 -07:00
Edward Lee
e94143f4e2 Bug 504212 - Have javascript stack traces show [object Object] for various CryptoWrapper functions
Pass the ID object instead of reading out the password and passing it in to encrypt/decrypt.
2009-07-14 16:51:04 -07:00
Edward Lee
2b33ab565b Bug 504177 - Don't block the UI when syncing up/down
Sync.sleep(0) to let the main thread do UI, etc. after every record encrypt or decrypt.
2009-07-14 14:01:26 -07:00
Edward Lee
84c52a6411 merge weave. resolve addition of kSpecialIds to bookmarks.js near removal of Async.sugar. 2009-06-23 22:34:57 -07:00
Edward Lee
ad5f01ba04 Bug 493816 - Timestamps need to be saved as string prefs
Internally store .lastSync as a string but keep exposing set/getters as float values. parseFloat takes both strings and numbers and gives a number.

--HG--
extra : rebase_source : b657dd3f732b9f3b3b0ef208ee53e799aa87d2c6
2009-06-16 17:22:59 -07:00
Edward Lee
4084603da5 Remove async.js and wrap.js and remove remaining references to Async.sugar, etc. 2009-06-06 18:24:30 -05:00
Edward Lee
1b781dda8c Remove Async.sugar from engines.js. 2009-06-05 15:34:39 -07:00
Edward Lee
48b4ac18ef Engine_wipeServer: async -> sync. SyncEngine__wipeServer: async/yield -> sync. 2009-06-05 15:34:37 -07:00
Edward Lee
f9aeec8b2c Engine_wipeClient: async -> sync. Engine__wipeClient: async/yield -> sync. 2009-06-05 15:34:35 -07:00
Edward Lee
3d3f786723 Engine_resetClient: async -> sync. SyncEngine__resetClient: async/yield -> sync. 2009-06-05 15:34:34 -07:00
Edward Lee
81f7652b56 Engine_sync: async -> sync. SyncEngine__sync: async/yield -> sync. 2009-06-05 15:34:32 -07:00
Edward Lee
4f22e547b2 Temporarily convert _notify to _notifyAsync and add a Utils.notify. 2009-06-05 15:34:23 -07:00
Edward Lee
563c0a0fd0 SyncEngine__processIncoming: async/yield -> sync. 2009-06-05 00:39:35 -07:00
Edward Lee
e7a503025a CollectionIterator_next: async + async/yield -> sync. 2009-06-05 00:38:23 -07:00
Edward Lee
3a4f2ca0ee Store_applyIncoming, SyncEngine__applyIncoming: async + async/yield -> sync. 2009-06-05 00:36:11 -07:00
Edward Lee
fce5aa36de SyncEngine__syncStartup, SyncEngine__uploadOutgoing, SyncEngine__syncFinish all async/yield -> sync. 2009-06-04 23:48:27 -07:00
Edward Lee
228f2e2568 CryptoWrapper_decrypt: async + async/yield -> sync. Fix up bookmark/client decrypt. 2009-06-04 19:06:57 -07:00
Edward Lee
7adf2ffa78 CryptoWrapper_encrypt: async + async/yield -> sync. 2009-06-04 17:58:37 -07:00
Edward Lee
a9de56fef0 CryptoMeta_addUnwrappedKey: async + async/yield -> sync. 2009-06-04 17:36:34 -07:00
Edward Lee
df16a63edb PubKeyManager_getDefaultKey: async + async/yield -> sync. 2009-06-04 17:04:51 -07:00
Edward Lee
ce37486d9c RecordMgr_get: async + async/yield -> sync. 2009-06-04 16:50:57 -07:00
Edward Lee
68ce1e4224 Change SyncEngine._reconcile to not be async/yield. 2009-06-04 14:18:04 -07:00
Edward Lee
1134960b32 Fix Resource.* call sites used for wiping the server and initial sync (+ key gen upload) 2009-06-04 10:17:27 -07:00
Edward Lee
428d54a14a Change Resource.get() to be sync (no callback) and fix up call sites used for login + sync now. 2009-06-04 10:16:07 -07:00
Edward Lee
232befb843 Bug 493256 - Reconcile doesn't handle removed items correctly. r=thunder
Have engines check if the deleted flag is the same for _isEqual.
2009-05-15 13:18:16 -05:00
Edward Lee
665e488e28 Bug 493001 - Dynamically generate weave prefs engine list based on registered engines
Remove existing hardcoded engine list + prefs and build checkboxes based on registered engines. Engines that give null for enabled (instead of true/false) are considered disabled. Update base Engine enabled get/set to give null and allow setting.
2009-05-14 11:38:15 -05:00
Dan Mills
76438f3876 skip getting meta records for records already in the meta list 2009-04-08 18:08:45 -07:00
Edward Lee
a7b212ab0b Bug 486810 - Engines (and their trackers) are created with every new window. r=thunder
Register the built-in engines on service start-up instead of from the overlay, and have Engines.register check if the engine has already been registered.
2009-04-08 14:39:14 -05:00
Edward Lee
75bacf714f Bug 487308 - Allow registering of an array of engines
Import engines to the Weave global object and use them to register engines, which checks if the arg is an array. To support handling of errors (unused), the engine is returned on register failure.
2009-04-07 16:45:41 -05:00
Edward Lee
cebb90b5bc Add a Utils.isArray and use it 2009-04-07 16:45:29 -05:00
Edward Lee
3993035a59 Bug 482670 - WBORecord should implement a .deleted property
Expose a .deleted property that engines can set to true to store an even thinner deleted payload (empty string "" instead of "[null]") on the server. Handle deserializing of deleted records by setting the property. Note: Engines must set their payloads to something JSON-able if it's not a delete record.
2009-04-03 12:38:47 -05:00
Edward Lee
4f6ebe6be5 Bug 481741 - Switch to JSON from nsIJSON 2009-03-31 14:09:38 -05:00
Edward Lee
75994bc68f Bug 482898 - Allow an optional array of engines for wipeRemote, wipeServer, resetClient, etc.. r=thunder 2009-03-20 00:13:16 -07:00
Edward Lee
e2f30efcf4 self.cb not this.cb for engine._wipeClient 2009-03-11 14:35:23 -05:00
Dan Mills
7b9eedde66 Bug 478327: Send minimal '[null]' payload for deleted records 2009-03-11 01:40:04 -07:00
Edward Lee
edda1db597 Bug 480457 - Expose a resetClient API for weave service and engines. r=thunder 2009-02-26 22:36:14 -08:00
Dan Mills
01ab4aa284 add some trace-level debugging to reconciler; ignore old (changed) IDs while reconciling 2009-03-03 01:29:35 -08:00
Dan Mills
3e1235a9e5 Switch away from __defineGetter__ due to bug 481104; explicitly cache logins in password engine; whitespace fixes 2009-03-02 18:55:26 -08:00
Anant Narayanan
0f6fdde06c Remove getter that fixes the 'two-store' problem 2009-03-03 01:15:48 +01:00
Dan Mills
bfa22eed1a Bug 480461: skip items that cannot be decrypted 2009-02-26 17:31:06 -08:00
Dan Mills
50621a1889 remove unused observer service getter 2009-02-26 13:52:12 -08:00
Dan Mills
edcaabd226 reset lastSync pref before setting it, as some clients have mismatched types saved 2009-02-23 19:33:40 -08:00
Dan Mills
8c671b69c9 Wrap.notify now takes the topic prefix as an argument, instead of requiring this._osPrefix to be set; use Observers.js in several places (sync.js, status.js, etc); some event topics have changed, beware; fix up status window, now prints some useful status as sync progresses 2009-02-22 00:04:58 -08:00
Dan Mills
1490d0c250 shorten engine logging pref names, set defaults for them 2009-02-20 01:51:20 -08:00
Dan Mills
d0e9169992 pass the cryptoMetaURL to the store's createRecord 2009-02-19 00:36:55 -08:00
Dan Mills
1883c83218 make sure to only save the last sync timestamp as a number 2009-02-18 20:17:30 -08:00
Dan Mills
a6a7141691 add a method for resetting the engine's lastSync pref 2009-02-17 13:20:02 -08:00
Dan Mills
a30d82e53d add '-engine' to engine name in observer notifications 2009-02-11 21:51:24 -08:00
Dan Mills
57c5368d7a remove sm-weave-proxy01 hack 2009-02-11 13:47:51 -08:00
Dan Mills
aafa72814f multi-cluster support; use new 'newer' flag instead of 'modified' for collection searches 2009-02-10 00:57:16 -08:00
Dan Mills
03d78b3d84 set record IDs inside the store; fix bookmarks tracker, it was ignoring change events (e.g., changed title and such) 2009-01-27 16:54:54 -08:00
Dan Mills
a47fcda5b1 only url-encode weave IDs when constructing a URL with them 2009-01-27 16:36:00 -08:00
Dan Mills
a3d9640e3c Make records not extend Resource, instead they have a serialize() method that can be used to create a Resource for uploading if necessary.
Use global service instances under Svc.*
Consolidate various record managers (caches/convenience factories) to extend the same base object
Log exceptions caught by notify wrapper
2009-01-27 13:35:10 -08:00
Dan Mills
84340dcdc2 cast modified timestamp to an int if it's a string 2009-01-21 19:02:38 -08:00