⌘ Development Progress Day 2

Now that UILabel is all set, I continue to hook other texts that are long enough to be "commanded", e.g. the UITableHeaderFooterView. There are also some reluctant UILabel that refuses to be hooked. For these I hooked their superviews (UINavigationBar & UIPickerTable) for extracting the text info.

And here comes the real challenge: hooking the UIWebDocumentView. Yes I knew the Clippy 0.95-5 experiment was failed. But this does not prevent anyone else from finding another way to achieve the same goal right?

My first naïve approach is to get the DOM node at the touch point, like this:

[[self webView] elementAtPoint:(touchPoint)];

and like Clippy 0.95-5, it crashes immediately at the 2nd touch. If I trace the class of the returned object*, I can see the class changes from e.g. DOMHTMLAnchorElement to NSCFType. In fact I encountered this before once when trying to implement setSelection() to <textarea/> and <input/> in ℏClipboard using the DOMRange object. It fails on the 2nd call. I call this the "observer effect". Right after you "observed" any DOM objects, the object will "collapse" and you can't reference it reliably anymore.

A workaround is to -retain it right after the DOM object is accessed. But this introduces memory leak. This is really a last-resort option.

And actually it turns out to be work done too complicated. There is a -[UIWebDocumentView approximateNodeAtViewportLocation:] method in the Interaction category which can easily be overlooked. It returns a DOMNode without any observer effect AFAIK. Except it crashes on 2.2 complaining ASSERTION FAILED: !WebThreadIsEnabled() || WebThreadIsLocked(). Well that's easy enough to solve. Just lock the WebThread() everytime we need to access this method.

So I used:

WebThreadLock();
DOMNode* activeNode = [self approximateNodeAtViewportLocation:&(touchPoint)];
WebThreadUnlock();

and done! The DOMNode is got and we can do anything we want. Too good to be true. Turns out the DOMNode will only be returned on any objects that have actions, i.e., <a>, <input/>, <textarea/>, <img/>, any thing that has onXXX events, but not static texts. If touchPoint is on a static text, nil will be returned.

But why this method prevents the observer effect? After lots of experiments, finally it turns out that it's the WebThreadLock() and WebThreadUnlock() doing the tricks. In fact, if I do

WebThreadLock();
[[self webView] elementAtPoint:(touchPoint)];
WebThreadUnlock();

then everything works perfectly! At least in the simulator. (Hey Ryan, if you're reading this, as now I've solved this problem, can you add Safari copying back in Clippy 0.95-6? :D)

So, at the end of Day 2, I can hook to any DOMNodes, besides UILabel, UITableHeaderFooterView, UINavigationBar and UIPickerTable.



Note: -[WebView elementAtPoint:] returns a subclass of NSDictionary. By the returned object I mean the value corresponding to the WebElementDOMNode key.