Friday, July 31, 2009

GriP, GriPGRIP & class-dump-z 0.1-11q Released

Starting from this version, Memory Watcher will not be included in GriP. Instead, it will be moved to a new package, "GriP Really Impressive Pack" (GriPGRIP). The distinction is that, GriP shall only contains the most needed extensions, i.e.
  • Mail
  • Push notification
  • SMS (by xshad0w, not included in 0.1-11q yet),
the 2 themes, Default and Double Height Status Bar (also by xshad0w, also not included yet), and 3 modal table themes, Default, HUD and Translucent Black (I may remove the last one too). On the other hand, GriPGRIP is a collection of all other stuff that will utilize GriP, like calendar events, remote Growl notifications, themes, etc. (The distinction is because GriP is growing to 1 MiB pretty soon. I want to keep the size down.)




Anyway, 0.1-11q is mainly an issue-fixing release, that include:
  • Issue 223. You can configure some GriP messages to wake the device up.
  • Issue 176. An new option so that GriP message can be expanded by default.
  • Issue 209. You've Got Mail will show the user name of the mail owner, instead of the address.
  • Issue 168. All apps can be selected shown in Game Mode. The UI is improved too :)
  • Issue 172. Multiple push notifications from the same app will be coalesced.
  • Suspension behavior can be updated properly.


If you have GriP 0.1-11p, you don't need GriPGRIP 0.1-11q, because there's nothing new.




And class-dump-z. There are 4 new features in this release:
  • The -N flag, which allows you to disable struct name prettifying (__CFArray* → CFArrayRef).
  • Optional properties. Such a thing does not exist directly, but can be inferred when its getter and setter are optional. (See the UITextTraits protocol for example).
  • The -h proto flag, which will hide all methods that's just adopting a protocol.
  • The -h super flag, which will hide all inherited methods.

Note that if you use the -h super flag directly on the Mac, it is almost certain that class-dump-z will crash. This is because to get the superclass info class-dump-z must open some framework bundles, and by default it will search in /System/Library/..., which certainly does not contain anything iPhoneOS-related. You have to provide the -y flag to specify the sysroot as well. For those who have installed SDK, you need to add

-y /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk

Thursday, July 30, 2009

Text input on 3.0, Part 4 — UIKBKeylistReference

  • When I was preparing this whole stuff many wrong directions were taken and generates quite a number of utilities that could be useful later. Who knows? Head to http://code.google.com/p/networkpx/source/browse/etc if you are interested.
  • You will need UIKB*.h to compile my examples. The headers can be found in my other project page at github. However, besides ActorKit no header there is actually complete (yet).





Hopefully it's the last of the series. I'm now making a convert from layout.plist to .keyboard files. This is a major foundation for backward-compatibility with previous keyboards, and also allows you to use custom keyboards on 3.0 without iKeyEx (but, without all the benefits like unlimited numbers of keyboards). The basic idea is described in Part 1.

Anyway...

Unlike the old UIKeyboardLayoutRoman, the new UIKeyboardLayoutStar is "scalable", that means this keyboard will still look sharp even if used on a 10" tablet PC. This is because the whole UIKeyboardLayoutStar is drawn at run time with Rounded rectangles and Gradients and Shadows and stuff. It's vector graphics. (In constrast, UIKeyboardLayoutRoman is precomposed into a PNG image.)

But even the layout has the potential to be scalable, all keyboards I've created before (using Star) is not, because I haven't told it what to scale. It's like an iPhone app, most of them can be easily presented in landscape mode, but if the programmer doesn't implement the -shouldAutorotateToInterfaceOrientation: method the app will always stuck in portrait mode.

As I've repeated many times, keyboard construction occupies 70% mass of UIKit, and that's roughly 8 MiB of code. There are 148 keyboards defined in UIKit so far, so each keyboard will be defined by 55 KiB of code and data. And unfortunately, most of these are code. Now, ARM's register-relative addressing mode allows a maximum span of 11 bits, i.e. 8 KiB, but there's 55 KiB of code! That means each single function must be chopped into chunks by the compiler. To the analyzing decompiler (ravel-arm), it is a difficult situation, because an unconditional branch at the end may mean a tail-function-call. The data after may be interpreted as code, and ruins all stored computation in between. This is happening in the keyboard-defining functions, so by static analyzing the code I can only get to how Apple defines keys up to the Capital Letter U (that's Q, W, E, R, T, Y, U), and then a branch across some mis-codified data, and all selector and class info are lost.

So? Initially I dumped all 148 keyboards and analyzed the resulting archive, but this doesn't tell us the implementation detail. Eventually I hooked objc_msgSend to give what's being called (since I knew only Obj-C functions are called and no structs are returned, I've ignored all other messaging variants).

The result is over 9000 bytes and I can't pastie it. But I can pastie part of the decompiled result. The UIKBKeylistReference part is what I was missing. Now let's plug in the keyboard, and we get... a dangling Q as we've expected.

But if we resize the keyboard from 320x216 to 240x216? Yes, the key is correctly scaled down:

And removing the references even make the dimensions wrong, so the scaling is really the reference in effect.

(When the reference is defined, the original geometry of the keys will be overridden).
(Also note that this keyboard cannot be scaled vertically, because the vertical measures of the rows are absolute values (pixels).)

One puzzling property of references is the "flag". Yes we know it must be bitfield of some sort, but what? If we decompile the class UIKBKeylistReference we get:
  • 1 = isKeysetReference
  • 2 = isKeylistReference
  • 4 = isKeysReference
  • 8 = isNamedKeyReference
  • 0x10 = isKeyIndexReference
  • 0x20 = isKeyIndexRangeReference
  • 0x40 = isGeometryReference
  • 0x80 = isAttributesReference

Apparently it refers to while the name elements interpreted.

Why we care about scaling? One, it allows us to define landscape and portrait keyboards with the same content (the keys), and to switch between the two we just need to change the outermost dimension, and the rest will be seamlessly changed. And two, who said the keyboard size must be fixed at 320x216 / 480x162? What if there is really an Apple tablet and uses the iPhoneOS? Or there's an iPhone mini? The resolution can't be the same, so the keyboard size must be altered as well. On 2.x we're forced to hard-code these 4 numbers because UIKeyboardLayoutRoman expects us to do so. Now having the better solution with UIKeyboardLayoutStar, I'd rather spend more time to take advantage of it and make iKeyEx more robust in the future.

Monday, July 27, 2009

Multiple Row Selection with UITableView — The Easy Way

Cocoa with Love introduced the hard way to implement multiple row selection for 2.x firmware. On 3.x, there is a much easier way to do the same, if you dare you embrace undocumented methods.

Multi-selection is regarded as one of the editing style. Therefore, to make a cell multi-selectable, implement this in your UITableViewDelegate:
-(UITableViewCellEditingStyle)tableView:(UITableView*)tableView editingStyleForRowAtIndexPath:(NSIndexPath*)indexPath {
...
return 3;
}

The "3" here means multiselection. The result is like this:


To get the selected rows, call the -indexPathsForSelectedRows method on the table view.
NSArray* selectedRows = [tableView indexPathsForSelectedRows];


If you dislike the red check mark, you can use the undocumented multiselectCheckmarkColor property to change it. Unfortunately, it has to be applied on the whole table.
tableView.multiselectCheckmarkColor = [UIColor blueColor];

The light blue background color cannot be changed unless you subclass or categorize UITableViewCell and override the -_multiselectBackgroundColor method, like this:
-(UIColor*)_multiselectBackgroundColor { return [UIColor yellowColor]; }

Saturday, July 18, 2009

class-dump-z & GriP 0.1-11p released

Downloads available in http://code.google.com/p/networkpx/downloads/list as usual.

What's new in GriP: Again, another attempt to fix issue 129.

What's new in class-dump-z:
  • Doesn't crash with WebKit.framework anymore.
    • The offending struct is XXStruct_GIci6C. Due to some strange thing in the GCC that compiles WebKit, the type of function pointer is encoded into "^" (pointer to !#$^*@#NO_CARRIER) instead of "^?" (pointer to some unknown stuff). The original class-dump won't handle this stuff either, and spit out a huge error string.
  • Doesn't crash with Symbolication.framework anymore.
    • The offending class is VMUCallTreeNode, which contains a union to a pointer to a static-typed objective-C class, which will cause a segfault in 0.1-11o. This was just an error in class-dump-z.
  • Forward declaration of some structs were missing if it is nested. Not anymore.
  • Fixed issue 187. It was because class-dump-z missed the LC_LOAD_WEAK_DYLIB commands.
  • Fixed issue 198.
  • Linux version: PCRE is static-linked. No need to search for the right version of libpcre.so anymore.

Thursday, July 9, 2009

Analyzing Objective-C's for-in loop (fast enumeration)

Objective-C 2.0 supports for-in loop like this:

for (XXSomeClass* obj in someArray) {
[obj doSomething];
}


it's nice to the programmer, but painful for the disassemblers because the simple loop above is translated into this bulky ASM:

            ldr               r1,[pc,#0xe8]
mov r3,#0x0
str r3,[sp,#0x44] // state
str r3,[sp,#0x48] // itemsPtr
str r3,[sp,#0x4c] // mutationsPtr
str r3,[sp,#0x50] // extra[0]
str r3,[sp,#0x54] // extra[1]
str r3,[sp,#0x58] // extra[2]
str r3,[sp,#0x5c] // extra[3]
str r3,[sp,#0x60] // extra[4]
ldr sl,[pc,r1]
add r3,r3,#0x10
str r3,[sp]
mov r1,sl
add r2,sp,#0x44
add r3,sp,#0x4
bl objc_msgSend (stub) ; [someArray countByEnumeratingWithState: objects:count:]
subs r5,r0,#0x0
beq loc_000100
ldr r3,[sp,#0x4c]
ldr r3,[r3]
mov r6,r3
loc_000094: mov r4,#0x0
b loc_0000a4
loc_00009c: ldr r3,[sp,#0x4c]
ldr r3,[r3]
loc_0000a4: cmp r6,r3
beq loc_0000b4
mov r0,r8
bl objc_enumerationMutation (stub)
loc_0000b4: nop
nop
nop
nop
nop
add r4,r4,#0x1
cmp r5,r4
bhi loc_00009c
mov r3,#0x10
str r3,[sp]
mov r0,r8
mov r1,sl ; "countByEnumeratingWithState:objects:count:"
add r2,sp,#0x44
add r3,sp,#0x4
bl objc_msgSend (stub) ; [? ?]
subs r5,r0,#0x0
ldrne r3,[sp,#0x4c]
ldrne r3,[r3]
bne loc_000094
loc_000100: ...


The disassembled code is
NSFastEnumerationState enumState;
enumState.state = 0;
enumState.itemsPtr = NULL;
enumState.mutationsPtr = 0;
enumState.extra[0] = 0;
enumState.extra[1] = 0;
enumState.extra[2] = 0;
enumState.extra[3] = 0;
enumState.extra[4] = 0;
NSUInteger count = 16;
id buffer[10];
do {
NSUInteger copiedItemsCount = [someArray countByEnumeratingWithState:&enumState objects:buffer count:count];
for (unsigned long i = 0; i < copiedItemsCount; ++ i) {
if (*(enumState.mutationsPtr) != 0)
objc_enumerationMutation(someArray);
[enumState.itemsPtr[i] doSomething];
}
} while(copiedItemsCount != 0);


What does the state and extra do? Up to the implementor. (And objc_enumerationMutation simply kills the program and reports that a mutation happened.)

Wednesday, July 8, 2009

Text input on 3.0, Part 3 -- Hooking

I think the best solution for now is put a hook at -[UIKeyboardLayoutStar setKeyboardName:appearance:]. If we see a custom name, we could put our own keyboard into the cache dictionary.

Who called -[UIKeyboardLayoutStar setKeyboardName:appearance:]? From the backtrace,
#1  0x30b16dae in -[UIKeyboardLayoutStar showKeyboardType:appearance:orientation:] ()
#2 0x30a1da78 in -[UIKeyboardImpl updateLayout] ()
#3 0x30a1a175 in -[UIKeyboardImpl setDelegate:force:] ()
#4 0x30a1815a in -[UIKeyboardImpl setDelegate:] ()

and the calling code of -[UIKeyboardLayoutStar showKeyboardType:appearance:orientation:] is
[self setKeyboardName:UIKeyboardGetKBStarKeyboardName(
UIKeyboardGetCurrentInputMode(),
orientation, keyboardType)
appearance:appearance];

and UIKeyboardGetKBStarKeyboardName is hard-coded to generate the expected keyboard name. But before we start we must ensure the input mode can be handled by UIKit! In 3.0 there's a function UIKeyboardGetSupportedInputModes() which returns the 46 hard-coded input mode names. Well, we could of course use MSHookFunction to replace it, but I'd like to see if it's possible to do it with Objective-C only, because MSHookFunction is not available on x86. And actually there is a trick — we see that UIKeyboardGetSupportedInputModes() returns a shared object, which is an NSMutableArray*. What does this mean? Well, we can call this function once on initialization, modify it, and from that point on UIKeyboardGetSupportedInputModes() will record our change.

Then we need to trick UIKit into thinking our keyboard uses UIKeyboardLayoutStar. This information is extracted in UIKeyboardInputModeUsesKBStar(), which is again hard-coded. However, this function uses an NSDictionary* which cannot be accessed outside. So we can't test on simulator? Not really, we could actually hook -[NSDictionary initWithObjectsAndKeys:] instead, and swap it back to the original method when done.

There are a few other hard-coded dictionary in UIKeyboardInputManagerClassForInputMode(), UIKeyboardLayoutDefaultTypeForInputModeIsASCIICapable(), UIKeyboardGetKBStarKeyboardName(), etc. After all these are replaced we can get a totally empty keyboard without error — a good start.

Now we can insert the actual hook to -setKeyboardName:appearance: and set the keyboard. This part is not difficult because there's are reference implementations, like that disassembly of getKeyboardiPhonePortraitQWERTY() and dump of various keyboard files.

Monday, July 6, 2009

Text input on 3.0, Part 2.

In the last part, we found that we may be able to use the .keyboard format via GetKeyboardDataFromBundle(). This function is only called from -[UIKeyboardLayoutStar setKeyboardName:appearance:], and is not exported, so it seems .keyboard really only contains the keyboard geometry.

To see if it really works, let's save an internal keyboard into a file:

[NSKeyedArchiver archiveRootObject:UIKBGetKeyboardByName(@"iPhone-Portrait-Arabic") toFile:@"/tmp/iPhone-Portrait-Arabic.keyboard"]

unfortunately, UIKBGetKeyboardByName is not an exported symbol, so we have to use nlist or lookup_function_pointers. Now, rename this file to e.g. iPhone-Portrait-QWERTY.keyboard (the English keybaord) and place it to UIKit.framework — and, indeed, the English keyboard becomes Arabic!

But how to create a UIKBKeyboard? A UIKBKeyboard has 3 major components: name (NSString*), visualStyle (NSString*) and keyplanes (NSArray*). The name is just the name you'd expect (iPhone-Portrait-Arabic), while the visualStyle must be one of @"iPhone-Standard" or @"iPhone-Alert". These are easy to construct. The meat are all in the keyplanes variable. The keyplanes is an array of UIKBKeyplane*. Difference keyplanes are switched between using the shift and 123/ABC keys. Again, each keyplane has a name (NSString*, e.g. @"Capital-Letters"); keylayouts, an array of UIKBKeylayout*; attributes, a UIKBAttributeList* and finally supported-types, an array of NSString*, etc., etc...

The content is summarized into http://code.google.com/p/networkpx/wiki/keyboard_file_spec. With the format analyzed, we can try to create our own keyboard using this new format.

Text input on 3.0, Part 1.

2 little notes:
  1. GriP 0.1-11o has been released to fix issue 129, and that's the only change.
  2. class-dump-z 0.1-11o has been released for Mac, iPhone and Linux. See this page for a comparison of other class-dumps.




The rest of this text is written at the same time I'm doing the reverse engineering. You may feel confused, because that's how researching goes.


The text input framework is greatly revamped in 3.0. The new UIKit, which is 5 times bigger than 2.0, has 70% of weight taken up by keyboard layouts. And the nontrivial letter-based input managers, once part of UIKit, is moved into its private framework, TextInput.

What was changed? As the first try, let's view the view hierarchy, although this time we can do it in compile time. From the class-dump we know the old UIKeyboard class is still there, so let's run the statement
UILogViewHierarchy( [UIKeyboard activeKeyboard] );
when the keyboard is present.

So we got this with the English keyboard:

<UIKeyboard: 0xd47980; frame = (0 264; 320 216); opaque = NO; layer = <CALayer: 0xd47d00>> ({{0, 264}, {320, 216}}, (null), #0, 3)
..<UIKeyboardImpl: 0xd3da20; frame = (0 0; 320 216); opaque = NO; layer = <CALayer: 0xd37b40>> ({{0, 0}, {320, 216}}, (null), #0, 3)
....<UIKeyboardLayoutStar: 0xd48650; frame = (0 0; 320 216); layer = <CALayer: 0xd4ba40>> ({{0, 0}, {320, 216}}, (null), #0, 3)
......<UIKBKeyplaneView: 0xd59250; frame = (0 0; 320 216); userInteractionEnabled = NO; layer = <CALayer: 0xd59300>> ({{0, 0}, {320, 216}}, (null), #0, 3)
........<UIKBKeyView: 0xd595e0; frame = (1 119; 40 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd596b0>> ({{1, 119}, {40, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59af0; frame = (279 119; 40 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59b60>> ({{279, 119}, {40, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59d10; frame = (1 173; 38 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59d50>> ({{1, 173}, {38, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59e20; frame = (41 173; 38 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59e60>> ({{41, 173}, {38, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd4dca0; frame = (81 173; 158 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd4dab0>> ({{81, 173}, {158, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd5bd10; frame = (241 173; 78 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd5bd50>> ({{241, 173}, {78, 42}}, (null), #0, 4)


In fact, all other keyboards — except the handwriting ones, and emoji also — now uses UIKeyboardLayoutStar to manager the layout. (The old UIKeyboardLayoutRoman is still there.)

The 6 UIKBKeyView are the special keys, as shown here:


The class dump of UIKeyboardLayoutStar shows some interesting members, e.g. m_keyboardName, m_keyplaneName, the list of dictionaries, etc. Take the Arabic keyboard as example.

  • m_keyboardName = @"iPhone-Portrait-Arabic"
  • m_keyplaneName = @"Letters"
  • m_states is a dictionary of 35 entries, with keys as UIKBKey to values as NSNumber (integers).
  • m_keyboard is the keyboard (UIKBKeyboard), which contains a lot of geometry info.


If we can replace UIKBKeyboard, then we can make our own layout. Where is that UIKBKeyboard set? Looking at the disassembly of UIKit (which is kinda painful now due to its huge size), we see the relevant part is at -[UIKeyboardLayoutStar setKeyboardName:appearance:]:

NSString* UIKeyboardGetCurrentInputMode();
NSBundle* UIKeyboardBundleForInputMode(NSString* mode);
NSData* GetKeyboardDataFromBundle(NSString* name, NSBundle* kbbundle);
NSBundle* _UIKitBundle();
UIKBKeyboard* UIKBGetKeyboardByName(NSString* name);

@implementation UIKeyboardLayoutStar
-(void)setKeyboardName:(NSString*)name appearance:(UIKeyboardAppearance)appr {
UIKBKeyboard* keyboard = [m_keyboards objectForKey:name];
if (keyboard == nil) {
NSData* keyboardData = GetKeyboardDataFromBundle(name, UIKeyboardBundleForInputMode(UIKeyboardGetCurrentInputMode()));
if (keyboardData == nil)
keyboardData = GetKeyboardDataFromBundle(name, _UIKitBundle());
if (keyboardData != nil) {
NSKeyedUnarchiver* keyboardArchive = [[NSKeyedUnarchiver alloc] initForReadingWithData:keyboardData];
keyboard = [keyboardArchive decodeObjectForKey:@"keyboard"];
[keyboardArchive finishDecoding];
[keyboardArchive release];
} else
keyboard = UIKBGetKeyboardByName(name);
}

if (keyboard == nil)
return;

// the rest are not interesting at the moment.
}
@end


So they defined 3 ways to fetch the UIKBKeyboard, (1) GetKeyboardDataFromBundle from the keyboard bundle (/System/Library/TextInput/TextInput_xx.bundle/), (2) GetKeyboardDataFromBundle from the UIKit bundle, and (3) UIKBGetKeyboardByName.

What does GetKeyboardDataFromBundle do? Basically,
GetKeyboardDataFromBundle (name, bundle) {
Go to the bundle's resource path;
Open ".keyboard";
Return the content of this file as NSData*.
}


But there isn't a single .keyboard file in the TextInput bundles! How about in UIKit.bundle? No, not even one. So the only path taken would be by UIKBGetKeyboardByName. And what does it do? Search from a function table with that name, and call that function. This probably explain the 70% mass of UIKit — these are functions to define the keyboards. Why they don't use the .keyboard files is a mystery, but from this it seems we can exploit the format to let iKeyEx run on it.