Bug 733372 - Fix an infinite recursion in GC marking; r=billm

Since the private data is sometimes packed in strange ways into the underlying
structure, we cannot just pass a reference to it to the marker.  Instead, we
store the gotten value on the C stack, do marking and potentially update its
value there, then pass the updated privates location to the object using
setPrivate.  If the setPrivate function triggers write barriers that end up
marking, this can recurse.  For now we should simply not do this.

--HG--
extra : rebase_source : ce2607907cc66f8beaeab433e21ba302cbf1b34e

js/src/jsapi.cpp

@@ -4267,7 +4267,7 @@ prop_iter_trace(JSTracer *trc, JSObject *obj)
          */
         Shape *tmp = (Shape *)pdata;
         MarkShapeUnbarriered(trc, &tmp, "prop iter shape");
-        obj->setPrivate(tmp);
+        JS_ASSERT(tmp == pdata);
     } else {
         /* Non-native case: mark each id in the JSIdArray private. */
         JSIdArray *ida = (JSIdArray *) pdata;