networkpx Project Blog

CrashReporter 0.2f available for testing; iKeyEx 0.3 sneak peak

2010-01-18T03:51:00.004+08:00

CrashReporter 0.2f has been available for testing. This version should fully eliminate the crash caused by fat binaries. Also, it has a hidden preference enabling "Email crash report to developer" within CR. But to avoid swarm of emails by stupid users blaming the tertiary suspects without any grounds you're required to read the source code of CR to know how to enable it ;).

Also, I have put up iKeyEx 0.3~beta1 for anyone who needs the new feature to test. 0.3 is an intermediate version between 0.2 and 0.5 to address some keyboard developers' requests, in particular:

When the key is a combining character, NFC is automatically applied.

variantType = "immediate-accents" (e.g. Thai and Arabic) is available via "text with traits".

more-after and shift-after (e.g. Zhuyin keyboard and the apostrophe) can be explicitly turned on and off via the "after" traits.

(0.5 shall support the "image" traits again, and thus keyboard theming; and the "star" operator in .cin IME.)

And, I'd like to point out that, if you want to change the text of Return and Space in your keyboard, you can already do so by strings.plist. This is supported from the start, not a new feature.

Two IDA Pro 5.x Scripts for iPhoneOS binaries

2010-01-06T05:51:00.002+08:00

If you use IDA Pro to disasseble iPhoneOS binaries (Mach-O/ARM), you may find these scripts useful:

dyldinfo: This script lets IDA Pro understands the DYLD_INFO[_ONLY] command introduced in firmware 3.1. Running this recovers some missing symbols.

fixobjc2: For binaries built with Objective-C ABI 2, this script can find and label all Objective-C functions and ivar offsets.

The result will look like this:

Because IDA doesn't allow "-" and spaces in names, the Objective-C function names are renamed like this:

-[Foo bar:baz:] → Foo.bar:baz:
+[Foo bar:baz:] → @Foo.bar:baz:
-[Foo(Cat) bar:baz:] → Foo(Cat).bar:baz:

(Note: Try not to run fixobjc2 before dyldinfo, some information cannot be found.)

iKeyEx 0.2b, CrashReporter 0.2e, class-dump-z 0.2a released

2009-12-29T01:52:00.005+08:00

Change log for iKeyEx 0.2b:

iKeyEx now uses postinst triggers to register layouts and input managers (r594). Keyboard makers may leave out the [pre|post][inst|rm] scripts now because the list will be updated as your keyboard is installed or removed. (You still need a postinst script if you want to support automatic purging.) As a side effect, all input modes explicitly deleted by the user (via Mix and Match) will be added back in and activated. If you really don't want that layout, uninstall it from Cydia.

Fixed issue 470: iKeyEx left/right arrows don't work in multi-line text input controls (r593).

No longer crashes for a layout with different landscape and portrait layout classes (r588). Thanks erik_joya for reporting!

Change log for CrashReporter 0.2e:

Fixed issue 481: CrashReport doesn't recognize FAT binaries. (r592)

Change log for class-dump-z 0.2a ~~(except Windows, I need to reinstall VS2008 first)~~:

Support for "hints file" (r589). See this wiki page for detail. Thanks rpetrich for suggestion!

iKeyEx and 5-Row QWERTY 0.2a will be on Cydia soon.

2009-12-15T01:44:00.002+08:00

The minor updates for iKeyEx (r579) and 5-Row QWERTY (r580) are released and will be on the Big Boss repo within 2 days. They can be downloaded from here as usual.

These updates are to address some defects reported after 0.2 is put on the repo, for iKeyEx:

Keyboards for Hebrew, Arabic and other right-to-left languages will default to use right-to-left input direction. (r578. Thanks DB42/OpenHebrew for reporting!)

Eliminated the built-in keyboard selection list in Settings, which became useless when iKeyEx is installed (it cannot be respected because the keyboard list is not safemode-safe otherwise). (r579)

For 5-Row QWERTY:

The , and . keys are swapped back to their expected position. (Thanks Optimo for noticing!)

Reset the special characters won't duplicate the double quote (") anymore. (r580)

Changing the layout flushes the cache correctly now. (issue 471)

iKeyEx 0.2~rc1, Cangjie 0.2-3, CrashReporter 0.2d released

2009-12-04T18:07:00.003+08:00

O hai, long time no post!

This is because I have posted many material which were to be put here to the "iPhone Development Wiki" created by Dustin Howett. (And also because of "schoolworks and tests" :p) This blog will remain for release notes only.

Back to the main topic, there are 3 major releases today: iKeyEx 0.2~rc1:

Different layouts in portrait and landscape orientation is supported again (r509, thanks DB42 for noticing)
CrashReporter support (r548)
Hooks only UIKit (r548).
Config file is now placed in ~/Library/Preferences/hk.kennytm.iKeyEx3.plist (r559).
Keyboard settings won't be lost due to Safe mode anymore.
.cin IME: Stop radical composition on failure by default.
.cin IME: Corrected sorting.
Other minor stuff.

Cangjie 0.2-3:

The shift key will be disabled in the 倉頡QWERTY鍵盤. If you want to enter English letters in this keyboard layout, long press on a key (e.g. 水 -> [水][E]).

CrashReporter 0.2d:

Made UIProgressHUD truly model to avoid crash due to accidental file deletion during symbolication.
Smarter suspect identification.
Symbolication in command line with the "-s" flag.
Complains if you did not install syslogd :)

Sorry Ollie Kett but...

2009-09-28T00:50:00.002+08:00

can I sell this scriptlet for $0.99?

javascript:var s=document.documentElement.outerHTML.replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").replace(/\n/g,"<br/>");document.open();document.write(s);document.close();

Disclaimer: This is not how Source Explore works. I did not even buy, try, pirate, or do anything with this app.

Compiling iPhoneOS (3.1) apps with Xcode 3.2 without Provisioning Profile

2009-09-26T02:05:00.005+08:00

Note: I don't know if this method still works, and I don't care. (And do not use the dd method mentioned in the article. It is extremely fragile to patch a binary file.)

Note 2: Please also check http://iphonedevwiki.howett.net/index.php?title=Xcode#Developing_without_Provisioning_Profile.

I want to compile.

This is easiest to solve. To compile you still need a certificate that can code-sign, but think this as a lip-service. Here is the procedure to create a self-signed code-signing certificate using Keychain Access. Make sure you create the certificate in the "login" (default) keychain, not the "System" keychain. After the certificate is created, perform these steps:

Open /Developer/Platforms/iPhoneOS.platform/Info.plist. (Backup if you want to be safe.)
Go to line 46. Replace the XCiPhoneOSCodeSignContext with XCCodeSignContext
Go to line 79. Replace the XCiPhoneOSCodeSignContext with XCCodeSignContext
Save the file.
Restart Xcode.
Compile!

After doing this, you can't use entitlements with Xcode anymore. But you have ldid -Sxyz.xml that does the same job.

The theory behind this is simple. When Xcode wants to code-sign, it has to consult DevToolsCore.framework on the procedures to code sign. The procedures are coded in the XCCodeSignContext class. The iPhoneOS placed more restrictions on the code-signing requirements, e.g. you must have a provision file. These extra procedures are provided in the class XCiPhoneOSCodeSignContext, as a plug-in at /Developer/Platforms/iPhoneOS.platform/Developer/Library/Xcode/Plug-ins/iPhoneOS Build System Support.xcplugin. Xcode isn't omniscience, so something must tell it that XCiPhoneOSCodeSignContext should be used, instead of the default XCCodeSignContext. This thing is the /Developer/Platforms/iPhoneOS.platform/Info.plist. In the past (iPhoneOS 2.x day), you add the PROVISIONING_PROFILE_[ALLOWED|REQUIRED] keys to give extra instructions to the plug-in, which it can promptly ignore. That's why eventually you need to take the dangerous way of binary-patching the plug-in. But why not just hide XCiPhoneOSCodeSignContext entirely? This is my hack's principle - tell Xcode to meet the new boss, same as the old boss.

I want to install and debug too.

(Note: I'm not fond of patching MobileInstallation.framework. In fact, on 3.1 there's no framework you can statically patch with.)

Unfortunately, installing is more complex than compiling because there are much more checking in the device side. Therefore the procedures will be so complex that ordinary developers should not follow ;).

Make sure you have ldid on Mac. (e.g. this). Place it in /usr/local/bin.

Create the file /usr/local/bin/ldid2. Make it executable. Fill it with:

#!/bin/sh

hasGTA=`expr "$*" : '.* -gta .*'`;
objpath=${!#}/`expr ${!#} : '.*/\([^/]\{1,\}\)\.app$'`;

if [[ $hasGTA == 0 ]]; then
 /usr/local/bin/ldid -S $objpath;
else
 TF=`mktemp -t x`;
 echo "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\"><plist version=\"1.0\"><dict><key>get-task-allow</key><true/></dict></plist>" > $TF;
 /usr/local/bin/ldid -S$TF $objpath;
 rm $TF;
fi;

Open /Developer/Platforms/iPhoneOS.platform/Developer/Library/Xcode/Plug-ins/iPhoneOS Build System Support.xcplugin/Contents/Resources/iPhoneCodeSign.xcspec
Replace the line saying CommandLine = "/usr/bin/codesign" with CommandLine = "/usr/local/bin/ldid2"
Now, copy /usr/libexec/installd from your device to your Mac.

Run this:

install_name_tool -change /usr/lib/libmis.dylib /usr/lib/libmiss.dylib installd
ldid -S installd

Now, create a file named libmiss.c, and enter these into the file:
```
extern int MISValidateSignature() { return 0; }
```

Compile libmiss.c to libmiss.dylib with gcc targeting iPhone. Make sure t you have all these flags:

-dynamiclib
-install_name /usr/lib/libmiss.dylib
-current_version 1
-compatibility_version 1
-Wl,-reexport-lmis
-flat_namespace

ldid -S libmiss.dylib, of course.
Copy the new installd to the device's /usr/libexec, and the libmiss.dylib to the device's /usr/lib.
Restart Xcode.
Install!

If you want to debug, add the -gta flag in "Other Code Signing Flags" in your Xcode Project.

OK, so WTF is going on? The first 4 steps allows you to debug the app on the device. The essence of this is replace codesign with ldid. Why this is necessary? Because of the change in Part 1, entitlements are no longer respected. But to debug you must have the get-task-allow entitlement. To fix it, we have to entirely replace codesign with our executable that adds the entitlement ourselves. ldid2 is created with this purpose.

The rest of the steps allows mobile installation to succeed without validation. When the app is installed from Xcode, "mobile_installation_proxy" is invoked, which spawned "installd". The validation part of installd is this:

 +00cd4 0000af2c 0010A0E3 loc_000cd4: mov               r1,#0x0
 +00cd8 0000af30 D19701EB             bl                MISValidateSignature (stub)
 +00cdc 0000af34 005050E2             subs              r5,r0,#0x0
 +00ce0 0000af38 0600000A             beq               loc_000d00
 +00ce4 0000af3c 54049FE5             ldr               r0,[pc,#0x454]                    ; -> 0xb398 "verify_executable"
 +00ce8 0000af40 5C149FE5             ldr               r1,[pc,#0x45c]                    ; -> 0xb3a4 "Could not validate signature: %x"
 +00cec 0000af44 0520A0E1             mov               r2,r5                             ; "_CodeSignature/CodeResources"
 +00cf0 0000af48 C9E3FFEB             bl                proc_00003E74
 +00cf4 0000af4c 0400A0E1             mov               r0,r4
 +00cf8 0000af50 B49701EB             bl                CFRelease (stub)
 +00cfc 0000af54 060000EA             b                 loc_000d1c

If we can force MISValidateSignature() to always return 0, any binaries will pass the test. This function is part of libmis.dylib, which is now part of the shared cache, so you can't binary patch this file. Replacing the implementation of a function is a perfect job with MobileSubstrate, unfortunately, no matter how I tried MS can't be injected. Therefore I use a trick: create a "proxy dynamic library" that changes only the MISValidateSignature function, and let the rest pass through. This is the purpose of libmiss.dylib. First, install_name_tool is used to repoint the libmis.dylib in installd to libmiss.dylib. Then, in libmiss.dylib we define a single function, MISValidateSignature, which returns 0 as we wanted. It linked with libmis.dylib, and reexport all the symbols so that the original symbols can still be found. But we have to use flat namespace to make sure our new MISValidateSignature can shadow the old one. Finally, there is a quirk in installd that required the dylib to be at version 1.0.0, so the -compatibility_version and -current_version flags are added.

Ouch these are all so difficult. Any easier methods?

Yes. Pay $99 to Apple. :p

GreenTea devices — You can still use Maps.

2009-09-25T12:21:00.005+08:00

I have forcefully enabled GreenTea on my device (with MobileSubstrate) but almost everything behaves normally (iTunes is still accessible, YouTube is not hidden, Maps shows everywhere, etc.). The only difference I've noticed is:

There is no Hybrid mode in Maps when GreenTea is on.

Oh well.

Anyway, the string "GreenTea" can be found in at least these 5 binaries: SpringBoard, MapKit, GMM, GraphicsServices, PhotoLibrary.

In SpringBoard, GreenTea can be found in these scenarios:

CFSTR("SBShowITunesStoreOnGreenTea") // in -[SpringBoard userDefaultsDidChange:]
kGSGreenTeaDeviceCapability near @selector(allowYouTube)

In MapKit,

-[UIDevice(MKAdditions) _mapkit_isChinaDevice] which returns GSSystemHasCapability(kGSGreenTeaDeviceCapability)
Some non-harmful changes in MKSearch***
In MKMaxZoomLevelForCoordinate(), if _mapkit_isChinaDevice returns true, jumps over the test against MKCoordinateIsInSouthKorea() // wtf is going on?!

In GMM,

Changes the server to http://www.google.cn/glm/...

In PhotoLibrary,

Something related to MMS.

China, no Google Maps for you (maybe)

2009-09-24T04:00:00.003+08:00

Today I was making the API for GraphicsServices, and found something... interesting.

Since 3.0 it was "well known" that there is a mysterious capability called as "Green Tea". Nothing was known except for this funny name. Things start to get clear in the 3.1 firmware. The GraphicsServices of 3.1 has a new set of API for querying some properties of Maps and MapKit. Strangely, in the disassembly of these functions, the mysterious "Green Tea" capability was refered, e.g.

Boolean GSMapKitUserShifting() {
  static CFStringRef gtDefault = CFSTR("MapKitUserShiftingGreenTea");
  static CFStringRef ngtDefault = CFSTR("MapKitUserShiftingNonGreenTea");
  if (GSSystemHasCapability(CFSTR("green-tea")))
    return GetMapsDefault(gtDefault);
  else
    return GetMapsDefault(ngtDefault);
}

It means the "Green Tea" capability is related to Maps, and takes different default values. No big deal right?

Moving downwards shows a more horrific picture, e.g. the GSMapKitAvailable() function is

Boolean GSMapKitAvailable() {
  if (!GSSystemHasCapability(CFSTR("green-tea")))
    return true;
  else {
    static CFStringRef defaultName = CFSTR("MapKitAvailableGreenTea");
    return GetMapsDefault(defaultName);
  }
}

Ah, when a device is "Not Green Tea", it directly says MapKit is available, but when the device is "Green Tea" it needs to check a default value? Sounds more like a restriction than a capability!

Further down, in GSMapsVisible(),

static Boolean _mapsLastVisible;
Boolean GSMapsVisible() {
  static Boolean _registeredListener = false;
  _mapsLastVisible = _GSMapsVisible();
  if (!_registeredListener) {
    CFNotificationCenterAddObserver(CFNotificationCenterGetDarwinNotifyCenter(), NULL, CarrierBundleChangedHandler, CFSTR("NewCarrierNotification"), NULL, 0);
    _registeredListener = true;
  }
  return _mapsLastVisible;
}

_GSMapsVisible() has a same "Green Tea" check as GSMapKitAvailable(). Strangely, it subscribes to the "New Carrier" Darwin notification, which probably means the Carrier can decide if Maps is visible or not (if com.apple.GMM.plist can be edited)!

So what does this "Green Tea" capability refers to? If we turn to GMM.framework, we can find the following code:

 +00244 0000ba5c             ldr               r0,[pc,r0]                        ; -> kGSGreenTeaDeviceCapability _mh_dylib_header
 +00248 0000ba60             ldr               r0,[r0]                           ; -> _mh_dylib_header
 +0024c 0000ba64             bl                GSSystemHasCapability (stub)
 +00250 0000ba68             ldr               r3,[pc,#0x1bc]                    ; -> 0xbc2c
 +00254 0000ba6c             add               r3,pc,r3                          ; GMM::Factory::isChinaDevice_
 +00258 0000ba70             subs              r0,r0,#0x0
 +0025c 0000ba74             movne             r0,#0x1
 +00260 0000ba78             strb              r0,[r3]                           ; -> GMM::Factory::isChinaDevice_

So having "Green Tea" is equivalent to China device! Let's recap:

Green Tea ⇒ Maps can be disabled.
Visibility of Maps is affected by Carrier
Green Tea = China Device

so a logical conclusion could be Carriers in China can affect the visibility of iPhone's Maps. Could it be a political decision? No matter what, China's iPhone already has no WiFi, but turns out it could be more crippled than expected.

class-dump-z 0.2-0 released.

2009-09-20T19:54:00.003+08:00

(The version jump is mainly because every other project is moving to 0.2 era.)

Download: http://code.google.com/p/networkpx/downloads/detail?name=class-dump-z_0.2-0.tar.gz.

Note: Mac OS X 10.6 is required in this version.

What's changed:

Universal binary is supported. You can choose different architectures with the -u switch. (Not -arch or --arch because I didn't use getopt_long.)
Completely recognizes the new __LINKEDIT format. XXUnknownSuperclass shall no longer appears for 3.1 binaries.
__attribute__((visibility("hidden"))) will be included as well when the class is not exported (e.g. UIKeyboardLayoutStar).
Options to hide categories and protocols.
Sort class alphabetically, but keep class methods and -init methods on top (suggested by ashikase)
Option to choose between +(void)foo; and + (void)foo;. (suggested by ashikase)
Fixes a minor bug where timeOut:(int)out was written instead of timeOut:(int)anOut.

About the LC_DYLD_INFO[_ONLY] command.

2009-09-20T03:14:00.004+08:00

With the introduction of the new __LINKEDIT format in iPhoneOS 3.1, many tools in the open toolchain are broken. This is all due to the unknown new commands LC_DYLD_INFO[_ONLY]. Although it's known to exist by many now, I found no useful documentation about this new format. Therefore, I'll outline what it is. Alternatively, you can study the source code of dyldinfo which contains every information here.

The LC_DYLD_INFO[_ONLY] commands

These load commands are numerically 0x22 and 0x80000022. The only difference between them are LC_DYLD_INFO_ONLY will abort loading when dyld doesn't understand the new format.

The structure of this load command has been described before. It refers to 5 chunks of data in the __LINKEDIT segment, which are called rebase, bind/weak_bind/lazy_bind, and export.

bind/weak_bind/lazy_bind

These 3 chunks are encoded with the same format. Please think of the data in these chunks as a tiny assembly language, which the only purpose is to "bind" (map) VM addresses to a symbol.

The encoding of each data is of the form:

Bit 7	6	5	4	3	2	1	0
opcode				imm operand				(extra data)

So there are at most 16 different opcodes can be used, and the immediate operand can hold a value 0 to 15. But most of the times a value >15, or even non-numeric data is needed. In these cases, extra data will be appended after this byte.

A large number is encoded in the "LEB128" format. In this format, each byte is separated into a "continue bit" (bit 7) and the "digits" (bit 0-6).

Suppose we want to encode the number 123456 in LEB128. Firstly, we write 123456 in binary, and separated into groups of 7 digits: 0000111,1000100,1000000. Then we insert the "continue bit" as 1, except the most significant one, which is 0 to signal the end of the number: 00000111,11000100,11000000. Finally, it should be in little endian, so we flip it around and write out the result: 0xC0 0xC4 0x07.

Apple so far defined 13 opcodes:

opcode	Symbol	Meaning
0	DONE	Finished defining a symbol.
1	SET_DYLIB_ORDINAL_IMM	Set the library ordinal of the current symbol to the imm operand.
2	SET_DYLIB_ORDINAL_ULEB	Same as above, but the library ordinary is read from the unsigned LEB128-encoded extra data.
3	SET_DYLIB_SPECIAL_IMM	Same as above, but the ordinary as set as negative of imm. Typical values are: 0 = SELF -1 = MAIN_EXECUTABLE -2 = FLAT_LOOKUP
4	SET_SYMBOL_TRAILING_FLAGS_IMM	Set flags of the symbol in imm, and the symbol name as a C string in the extra data. The flags are: 1 = WEAK_IMPORT 8 = NON_WEAK_DEFINITION
5	SET_TYPE_IMM	Set the type of symbol as imm. Known values are: 1 = POINTER 2 = TEXT_ABSOLUTE32 3 = TEXT_PCREL32
6	SET_ADDEND_SLEB	Set the addend of the symbol as the signed LEB128-encoded extra data. Usage unknown.
7	SET_SEGMENT_AND_OFFSET_ULEB	Set that the symbol can be found in the imm-th segment, at an offset found in the extra data.
8	ADD_ADDR_ULEB	Increase the offset (as above) by the LEB128-encoded extra data.
9	DO_BIND	Define a symbol from the gathered information. Increase the offset by 4 (or 8 on 64-bit targets) after this operation.
A	DO_BIND_ADD_ADDR_ULEB	Same as above, but besides the 4 byte increment, the extra data is also added.
B	DO_BIND_ADD_ADDR_IMM_SCALED	Same as DO_BIND, but an extra imm*4 bytes is also added.
C	DO_BIND_ULEB_TIMES_SKIPPING_ULEB	This is a very complex operation. Two unsigned LEB128-encoded numbers are read off from the extra data. The first is the count of symbols to be added, and the second is the bytes to skip after a symbol is added. In pseudocode, all it does is: for i = 1 to count define symbol offset += 4 + skip end for

For example, we want to bind the address 0x2020 (of the __DATA section, starting at 0x2000) to the symbol _XXHello, which is defined in the 9th loaded dylib, Hello.dylib. We would perform the following operations:

SET_DYLIB_ORDINAL_IMM(9)
SET_SYMBOL_TRAILING_FLAGS_IMM(0, "_XXHello")
SET_SEGMENT_AND_OFFSET_ULEB(2, 0x20)  ; usually __DATA is the 2nd segment.
DO_BIND()

So in binary it will be

0x19 0x40 "_XXHello\0" 0x72 0x20 0x90

rebase

I don't think rebase is useful, and rebase uses a similar approach to code rebase info as bind, so I'm ignoring it here.

export

Unlike bind, export is an entirely different beast. The content of the export chunk defines a trie, or a prefix tree. A node in this trie is encoded as:

Node = «uint8_t terminal_size» [Terminal] «uint8_t child_count» [Child] [Child] [Child] ...
Child = «char* suffix» «uleb128 offset»
Terminal = «uleb128 flags» «uleb128 address»

Known flags are:

1 = THREAD_LOCAL
4 = WEAK_DEFINITION
8 = INDIRECT_DEFINITION
0x10 = HAS_SPECIALIZATIONS

For example, if a dylib exported _XXHello at 0x1022 and _XXWorld at 0x1064, and _XXHelloWorld2 at 0x1558. A trie that represent these symbols would be:

_XX - [Hello] - [World2]
   \
     [World]

So we encode our root node as

00 01 "_XX\0" (offset to _XX)

and _XX as

00 02 "Hello\0" (offset to Hello) "World\0" (offset to World)

if we place the _XX node right after the root node the offset would be 7, so the root node is

00 01 "_XX\0" 07

The offset of the rest can be obtained like this. Now for the Hello node, since it defined a symbol, we have to fill in the Terminal info:

05 00 A2 20 /*=0x1022*/ 01 "World2\0" (offset to World2)

etc.

QuickScroll 2.2a should be available on Cydia soon.

2009-09-18T22:16:00.003+08:00

QuickScroll ~~2.2~~ 2.2a is released, and this marks the completion (so far) of the QuickScroll project. Thanks to the gdb for 3.1 I've finally squashed the "HiCalc" bug. I have already submitted it to BigBoss and should be available in a day. If you can't wait, you can still download from here.

Change log from 2.1a:

The scrolling indicator is no longer visible when scrollbar is used.
You should be able to use the scroller in HiCalc and other apps that canCancelContentTouches.

Change log from 2.2:

Fixed an obscure bug that causes crashing when the scroll view disappears. Thanks Optimo for discovering.

Porting the SDK's gdb for 3.1

2009-09-18T19:39:00.007+08:00

The gdb on Cydia currently doesn't work on 3.1 because of the new Mach-O format. We could compile gdb from source (which I've failed to do so), wait for someone else to compile from source (~~which I'm still waiting~~), or just use the gdb in the SDK.

Update: saurik has updated the gdb package that works in 3.1, which will be available tomorrow. You can download it now from http://apt.saurik.com/debs/gdb_1128-8_iphoneos-arm.deb.

The SDK's gdb is in /Developer/Platforms/iPhoneOS.platform/Developer/usr/libexec/gdb/gdb-arm-apple-darwin. While you can run it directly on your Mac, it is in fact a fat binary with 3 architectures:

file gdb-arm-apple-darwin

gdb-arm-apple-darwin: Mach-O universal binary with 3 architectures
gdb-arm-apple-darwin (for architecture ppc): Mach-O executable ppc
gdb-arm-apple-darwin (for architecture i386): Mach-O executable i386
gdb-arm-apple-darwin (for architecture armv5): Mach-O executable arm

The ARMv5 portion is what we want. But this gdb is useless to run on the iPhoneOS because it lacks all the essential entitlements. While entitlements can be inserted using ldid, there is a limitation needs to be worked-around: ldid doesn't support the armv5 architecture. We have to modify the source code of ldid to allow it:

@@ -557,6 +557,7 @@
                     case 12: switch (framework->cpusubtype) {
                         case 0: arch = "arm"; break;
                         case 6: arch = "armv6"; break;
+                        case 7: arch = "armv5"; break;
                         default: arch = NULL; break;
                     } break;

(I have also lipo -thin-ed the gdb before ldid-ing to make everything smooth.) After applying this patch, the ldid should recognize armv5 correctly. Now, save this portion of text as an XML file (e.g. gdb.xml):

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>com.apple.springboard.debugapplications</key>
 <true/>
 <key>get-task-allow</key>
 <true/>
 <key>task_for_pid-allow</key>
 <true/>
 <key>run-unsigned-code</key>
 <true/>
</dict>
</plist>

and then run:

ldid -Sgdb.xml gdb

, and the gdb should now be completely usable on the iPhoneOS.

iKeyEx 0.1-99j released, now works in 3.1

2009-09-17T02:13:00.002+08:00

Download: http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.iKeyEx3-0.1-99j.deb

This version is an emergency release to make it work in 3.1 (r497, r498). Along with it there are also these changes:

Fixed a crash when uninstalling the Chinese Phrase Tables package (r491)
Fixed a crash when phrases longer than 2 characters are processed (r499)
Fixed a bug in characters sorting, that the sorted array is incorrect (r499).

QuickScroll 2.1a.

2009-09-16T03:19:00.004+08:00

Download: http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.quickscroll2-0.2-1a.deb.

New features:

QuickScroll can be disabled for each particular app, including SpringBoard.
Activate by scrolling. (This is off by default since you can't have smooth scrolling with QuickScroll. Oh you have 3GS? Forget I what say then...)
New icon design by Sagitt.
Localizations.

After this release all features will be frozen. That is, no new features will be accepted, no matter how ingenious it is. (Ya know, this thing gotta be pushed out some day). Only bug fix and new localizations will be allowed for 2.2 (the public version).

QuickScroll 2.1.

2009-09-15T02:26:00.002+08:00

Download: http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.quickscroll2-0.2-1.deb

Changes:

The scrollers should now be easier to grab and move around.
Duration now really defaults to 2 seconds on first install. Thanks fusen for noticing.
Scrollbars can be chosen to jump on spot instead of next page. Thanks Sagitt for suggestion.

(If there're no other bugs I'll submit this version to BigBoss.)

QuickScroll 2 released

2009-09-14T01:37:00.002+08:00

With a 333% size increase*, QuickScroll 2 is released to improve the scrolling experience.

QuickScroll 2 now introduces scrollbars, which is the default.

Besides PDF files, scroll views that explicitly allowed for paging can also be targeted.

The old scroller is still accessible, but you can now move it around (and occupies much less space).

(To jump to a page, tap the 123 icon at the lower right corner in scroll bar mode, or tap the ← arrow button in scroller dialog mode.)

These configuration can be set in Settings.

As you can see, there are 2 more gestures you can choose. The two-finger tap should allow you activate QuickScroll in a table view easier.

I've eliminated the close button. The scrollers will disappear in 2 seconds of inactivity.

QuickScroll's scrollers are now actually a subview of the scrolling view, while in the 1st version it is an alert box. This change allows QuickScroll to be used in very high-level windows like those in SBSettings and GriP.

I don't know how to localize PreferenceLoader entries yet. So no localizations in this version, sorry.

*: 24 KiB → 104 KiB on disk.

Get UIView hierarchy, take 3.

2009-09-11T22:22:00.002+08:00

The previous post about dumping UIView hierarchy is actually over-complicated. Actually all you need is one command (in gdb):

po [[[UIApplication sharedApplication] keyWindow] scriptingInfoWithChildren]

The result will be very verbose, make sure your terminal has enough scrollback.

iKeyEx 0.1-99i released.

2009-09-10T03:11:00.003+08:00

Download = here.

Changes:

Fixed issue 313. In an alt (numbers) plane, pressing the space key will go back to the main (alphabets) plane.
Typing the apostrophe (') in the main plane no longer auto-switch to the alt plane.
Fixed issue 312, and many other auto-shift related quirks.
.cin IME:
- Number of candidates is limited to avoid near-infinite loop. 64.0 candidates should be enough for everyone.
- Multi-radical continuation works again.
- Fixed cases where blank candidates appear.
- Candidate searching now operates in serial for reliability. You may experience some degrade in performance.
- A progress indicator is added when the Patricia tree dump for the IME was first generated. This is essential for some huge IME like 輕鬆輸入法, which takes nearly 2 minutes for the first launch.

(Chinese Users: 倉頡輸入法及額外字頻表及詞庫亦更新至 0.2-1 版，這與 0.2-0 版內容上其實沒分別，只是製作 deb 時改用了 gnutar，從而避免因含非 ASCII 檔名而導致安裝失敗。若果你正在使用這些軟件，則不用更新。)

By the way, if you find any bugs, please report at Issues in the project page. If you leave a comment here or the wiki I can't guarantee I can dig and fix that.

Preemptive Warning: Comments not related to this content will be ignored.

Introducing Subjective-C, an objc_msgSend[_[st|fp]ret]? logger.

2009-09-07T02:17:00.006+08:00

Time ago I logged calls to objc_msgSend to understand how to construct UIKBKeyboards. But that logger is known to cause problems due to asserting the arguments use less than 1024 bytes. I needed to log calls again for issue 312, but the old buggy behavior leads me to rewriting it more reliably.

The result is the dynamic library called Subjective-C. It has the following new features:

Stack-safe. No arguments will be lost due to this logger.
Call tree construction.
Filtering.

along with the old features:

Print and format all arguments, and the return value.

Note:

Due to licensing, only the ARM version is released, although the x86 version works perfectly.
If your product depends on Subjective-C (why?), please note that it is GPLv3.
(No, it won't help even if I BSD everything.)

Sample output


 +[UIScroller _registerForNotifications]
 +[NSString alloc] {
  +[NSString allocWithZone:] (0x0)
 }
 +[NSBundle mainBundle] {
  -[NSRecursiveLock lock] <0x1007540>
  -[NSRecursiveLock unlock] <0x1007540>
 } = <NSBundle 0x100db50>
 -[NSBundle bundleIdentifier] <0x100db50> {
  -[NSBundle infoDictionary] <0x100db50> {
   -[NSBundle _cfBundle] <0x100db50> = 0x1009d60
  } = <NSCFDictionary 0x100af10>
  -[NSCFDictionary objectForKey:] <0x100af10> (@"CFBundleIdentifier") = @"com.yourcompany.Untitled4"
 } = @"com.yourcompany.Untitled4"
 -[NSPlaceholderString initWithFormat:] <0x100cf70> (@"%@.UIKit.migserver") {
  -[NSPlaceholderString initWithFormat:locale:arguments:] <0x100cf70> (@"%@.UIKit.migserver", nil, "∞≠") {
   -[NSCFString respondsToSelector:] <0x100adb0> (@selector(descriptionWithLocale:)) {
    -[NSCFString class] <0x100adb0> = NSCFString
    +[NSCFString resolveInstanceMethod:] (@selector(descriptionWithLocale:)) = NO
   } = NO
   -[NSCFString description] <0x100adb0> = /*self*/ @"com.yourcompany.Untitled4"
  } = @"com.yourcompany.Untitled4.UIKit.migserver"
 } = @"com.yourcompany.Untitled4.UIKit.migserver"

API

You can find the header file in subjc.h.

Most of the case you just need to call two functions:

void SubjC_start(FILE* f, size_t maximum_depth, bool print_arguments, bool print_return_value)
- Starts logging all objective-C calls. The result will be written to the file f, up to a maximum call-tree depth maximum_depth. You may use the boolean parameters print_arguments and print_return_value to control whether to format the arguments and return values. Not doing so may save some time.
void SubjC_end();
- Stops logging. This function must be called at the same level as SubjC_start.

The SubjC_initialize() function does the initialization things. It will be called automatically in SubjC_start if not done, so you don't need to explicitly call it.

The rest are to control whether a (sub)message can be logged. If a message is filtered, all calls spawned by
this message will not be logged. For example, if the selector copy is filtered, then in the log you will only see

-[XXSomeClass copy] <0xfedcba98> = <XXSomeClass 0x12345678>

instead of

-[XXSomeClass copy] = {
 -[XXSomeClass copyWithZone:] <0xfedcba98> (0x0) = <XXSomeClass 0x12345678>
} = <XXSomeClass 0x12345678>

Known issues

Subjective-C is not thread-safe. ~~I repeat: Subjective-C is not thread-safe. If there are ≥1 threads that uses Objective-C during logging you're almost surely doomed to crash.~~ OK it won't crash now by changing the global lr stack to thread-local, but it's still not completely thread-safe.
Variadic arguments can never be logged.
+initialize messages cannot be logged. This is because Subjective-C cannot work if the class is not initialized, so it implicitly initialize the class before logging.

How it works

All Objective-C calls (except the manually cached ones) will go through objc_msgSend, objc_msgSend_stret, and in x86, objc_msgSend_fpret. Therefore, if we replace these functions with our custom one the every message can be logged.

The tricky part is to call the original function after logging is finished. GCC has __builtin_apply_args and friends to construct the calls, but it assumes you know the maximum bytes of the arguments, because the approach taken by these is to copy the content of stack. This leads to the buggy behavior in the old logger.

To avoid this, the new logger was written in assembly. The approach (and the name) was inspired by saurik's Aspective-C (thanks for informing this work). In the new logger, every action before calling the original function was done without modifying the stack pointer and the content above it, nor the registers.

That means, in ARM, the registers r0-r3 cannot be modified, push/pop cannot be used, and the only free register is r12. Fortunately, heap memory can still be accessed, and well-defined functions will not affect anything except r0-r3, r12 and lr, so the only necessary step to perform is

Load the address of an array of 32-bit integers
Store r0-r3 in there
Call any functions we like
Load the address of that array again
Restore r0-r3 from there.
Call original objc_msgSend
Print the returned value and return.

But to return, the link register (lr) need to persist. And it needs to persist throughout the whole replaced function, which in the middle a self-call may happen. This means the lr must be stored on a stack:

Load the address of an array of 32-bit integers
Store r0-r3 in there
Push lr onto a custom stack
Call any functions we like
Load the address of that array again
Restore r0-r3 from there.
Call original objc_msgSend
Pop lr from the custom stack
Print the returned value and return.

The situation is easier on x86. Every argument, and the return address must be on the stack. Therefore the "Store r0-r3" steps can be ignored. Nevertheless, we need to replace the return address to ensure the original objc_msgSend return to our place, so the custom stack is still needed.

Compiling Subjective-C

To compile Subjective-C you need the following:

g++
Boost C++ library (for unordered_set)
Source code for the whole hk.kennytm.Peace subproject (for argument formatting).
APELite, if you compile for x86.
MobileSubstrate, if you compile for ARM.

subjc.mf contains the Makefile for ARM, and subjc.x86 for x86.

stmXX

2009-09-02T03:31:00.004+08:00


static int rx_reserve[8];

...

 __asm__(" mov r1, #1\n"
   " mov r2, #2\n"
   " mov r3, #3\n"
   " ldr r0, (reserve)\n"
   " mov r4, r0\n"
   " ????? r0!, {r1-r3}\n"
   " str r0, [r4, #16]\n"
   " b after_data\n"
   "reserve:\n"
   " .long _rx_reserve+12\n"
   "after_data:\n");
 
 printf("%d %d %d [%d] %d %d %d;\ndelta = %d\n",
     rx_reserve[0], rx_reserve[1], rx_reserve[2], rx_reserve[3], rx_reserve[4], rx_reserve[5], rx_reserve[6],
     rx_reserve[7]-(int)(rx_reserve+3) );

?????	Result
stmia	0 0 0 [1] 2 3 0; delta = 12
stmib	0 0 0 [0] 1 2 3; delta = 12
stmda	0 1 2 [3] 0 0 0; delta = -12
stmdb	1 2 3 [0] 0 0 0; delta = -12

iKeyEx & 5-Row QWERTY 0.1-99h are released

2009-09-01T06:57:00.002+08:00

Downloads can be found in here as usual.

Changes from "g" are:

You can now long-press control keys (left, right, etc) to repeat actions.
ANSI and X11 apps for control keys are now correctly detected.
The config file is moved to ~/Library/Keyboard/iKeyEx::config.plist. This allows the config to be preserved even after firmware upgrade. (Permission problems will also be fixed during installation.)
PSBundle for layouts and IMEs. Normal users can find them in Settings → iKeyEx → Customize.
In the delete cache page, the total file size of the cache entry will be reported.
Candidate calculation in .cin IMEs now actually runs in background.
Associated phrases (aka Completion) can be disabled.
iKeyEx-KBMan now registers input modes correctly without causing crashes. Also it now purges layout cache correctly.

The "h" version of iKeyEx is considered a release candidate. I'll try to get it to BigBoss's beta repo if no major bug is found.

Many of the changes in "g" and "h" are to prepare for the 5 Row QWERTY layout. Of course, the major change for 5 Row QWERTY is it works on 3.0, but even compared with 0.1-9b, there are a few points to need to notice:

You'll find that the Tab, Esc, Page Up keys etc become words instead of symbols. This is because, with the system fonts all the previous symbols cannot be rendered. For consistency I just change them all into words.
The "Autocorrection" part of the old pref bundle is now handled by Mix & Match.
Sometimes your customization won't take effect. Try to Delete cache if that happens.

There are no modifications other than these.

iKeyEx 0.1-99g is released

2009-08-30T21:36:00.002+08:00

Download.

Changelog:

“Text with traits” is now supported, that means your can add color, change font and text size for each key. However, you still cannot use key with image.
Native control keys support. (Left, right, home, page down, etc.)

Easier way to get UIView hierarchy.

2009-08-26T05:14:00.004+08:00

Before I documented how you can obtain the UIView hierarchy in syslog. It uses custom function, but there is are 2 built-in way to get this.

The first, standard way is to send a GSEvent type #500 to the application, then the dump will be written to /tmp/UIDump in plist format. You can achieve the same by calling in gdb:


call (void)[[UIApplication sharedApplication] _dumpUIHierarchy:0]

When the dump is completed, a Darwin notification "com.apple.UIHierarchyDump.finished" will be posted.

(You can also take a screenshot with _dumpScreenContents:0 / GSEvent type #501, but the file is in JPEG and pressing Lock+Home isn't that hard...)

(Note: May fail for AppStore apps due to sandboxing.)

GSEvent Recording and Playback in 3.0

2009-08-26T02:37:00.003+08:00

iPhoneOS has built-in API for application macro since 2.x. (Of course it was never documented.) (And the API was really changed in 3.0 to adopt for multiple recorders.)

For example, to record everything you've done during the for yourUIApplication:


 recorder = [Recorder new];
 [yourUIApplication _addRecorder:recorder];
 [recorder release];

The command _addRecorder: adds the object to the application's event recorder array. You can remove your recorder at anytime with


 [yourUIApplication _removeRecorder:recorder];

An event recorder must conform to the informal protocol


@protocol UIEventRecorder
-(void)recordApplicationEvent:(NSDictionary*)event;
@end

Here, "event" is a GSEvent converted to a plist using GSEventCreatePlistRepresentation(). You can add the event to an array and save that array for later reference, e.g.


@interface Recorder : NSObject { NSMutableArray* eventList; } @end
@implementation Recorder
-(id)init { if ((self = [super init])) eventList = [NSMutableArray new]; return self; }
-(void)save { [eventList writeToFile:@"events.plist" atomically:YES]; }
-(void)recordApplicationEvent:(NSDictionary*)event { [eventList addObject:event]; }
@end

Now say you want to play back the events you've previously collected. Just use this code:


NSArray* eventList = [NSArray arrayWithContentsOfFile:@"events.plist"];
float playbackRate = 1;
[app _playbackEvents:eventList atPlaybackRate:playbackRate messageWhenDone:target withSelector:@selector(done:)];

The higher the playback rate, the faster the system will run the events.

When the playback is finished, the "done:" selector will be called for "target". This selector must have a signature of


-(void)done:(NSDictionary*)detail;

the "detail" argument contains exactly 1 key, "UIApplicationEventRecordingDeliveryTimeUserInfoKey", which points to an array of the time the corresponding event happened.

Decompressing .zip files with iWorkImport.framework.

2009-08-25T05:25:00.002+08:00

.zip files need to be supported in lots of places on the iPhoneOS. .ipa are zip files, .docx are zip files, .key are zip files. Unfortunately there is no centralized framework or library to handle these zip files. In fact, all Bom.framework, OfficeImport.framework and iWorkImport.framework have their own, private implementation to decompress a .zip file. Out of the three, iWorkImport.framework is the friendliest.

This example code shows how to open a .zip file and read the content:

http://pastebin.com/f16274c11

(Of course, if you write your new application, you should not unzip like this :) use something documented such as ziparchive.)

Trivia for Preference Bundles.

2009-08-24T04:01:00.003+08:00

1. How to create that red delete button?

The red delete button in VPN is in fact very easy to implement. All you need to do is add the following code:


#import <UIKit/UIPreferencesDeleteTableCell.h>
@interface PSDeleteTableCell : UIPreferencesDeleteTableCell @end
@implementation PSDeleteTableCell
-(void)setValueChangedTarget:(id)target action:(SEL)action userInfo:(NSDictionary*)info {
 [self setTarget:target];
 [self setAction:action];
}
-(UILabel*)titleTextLabel {
 UILabel* res = [super titleTextLabel];
 res.textColor = [UIColor whiteColor];
 return res;
}
@end

and then in the specifier plist, modify your button as:


...
{ cell = PSButtonCell;
  action = nukeFromOrbit;
  label = "Nuke from Orbit;
  ...
  cellClass = PSDeleteTableCell;
},
...

The result is:

2. PSEditableListController

Normally list controllers inherit from the PSListController, but you can make a class inherit from PSEditableListController so that it becomes... editable! Actually, more like deletable. There will be an extra Edit/Done button on the top right hand corner, pressing it will reveal the deletion control. If you actually delete a cell, the corresponding specifier's deletionAction will be called.

3. A list of editing panes

There are several editing panes in Preferences.framework and .app. To use those panes, just use the specifier plist


{ cell = PSLinkCell;
  detail = PSDetailController;
  pane = PaneName;
  ... }

PSMultiValueEditingPane (Include just a PSSegmentCell, useless by itself.)

PSTextEditingPane (A pane containing a text field and a keyboard.)

RegulatoryPane (A static image showing the device is FCC approved and in other countries as well.)

LegalMessagePane

TimeZonePane (Time zone selector. The returned value doesn't contain just time zone, but also city location.)

RingtonePane (Note that the ringtone will be played when selected.)

iKeyEx 0.1-99f Released, now supports .cin IME.

2009-08-23T01:37:00.002+08:00

Download: here

Changelog: There were 3 versions in between, "d" was mainly to fix issue 1, "e" was an emergency release to fix a crashing bug on the preferences, and "f" fixes rest of the bugs and improves performance of characters tables.

- Fixed issue 1. .cin input manager can be used natively with iKeyEx.
- You can now the method to invoke the keyboard chooser, the disable it completely
- Fixed a bug that causes crash when a layout.plist using text with traits is encountered. Thanks DB42 (of OpenHebrew) for reporting!
- UI for cache removal.
- Basic localization (en, es, zh_TW, zh_CN).
- Fixed a bug that may cause preferences to crash. Thanks @mikitomo for reporting!
- Loading characters table is now much faster (from O(n) to O(1)), that the second time you use a .cin input method will load instantly, instead of need to wait for 1 ~ 3 seconds (at a price of ~200 KiB disk space).
- Name for layout.plist keyboard cache are now correctly generated. Thanks DB42 for reporting!
- Keyboard list will be properly updated after Mix and match.
- The counter in "Delete cache" will be properly updated after a cache entry is removed.
- Candidate list computation won't lock up the application forever (at worst case) anymore.
- Support for layout.plist on non-default keyboard type is fixed. Thanks DB42 for reporting!

Actually issue 1 was a major reason why the iKeyEx project exists, and now it is finally implemented. The .cin input method is mainly for use in table-based Chinese input method, but it can be used to implement that things too, e.g. the MultiTap method + Text prediction can be made on top of a .cin IME. The technical detail of .cin files and the technology around it can be found here.

The IME support is nowhere versatile. The .cin IME in iKeyEx so far only supports:

Prefix searching (like autocompletion)
Phrase completion

In the future it may also support:

Key charges
Dynamic dictionaries.

Embedding a preference bundle, in your app.

2009-08-12T20:14:00.003+08:00

If you want to use Preferences.framework in your app, you can do it like this.

First, make a PSListController subclass as you would when creating a Preference Bundle. Then create a subclass of PSRootController, with code like this:

@interface MyRootController : PSRootController
@end
@implementation MyRootController
-(void)setupRootListForSize:(CGSize)size {
 PSSpecifier* spec = [[PSSpecifier alloc] init];
 spec.name = @"Anything";
 
 MyListController* root = [[MyListController alloc] initForContentSize:size];
 root.rootController = self;
 [root viewWillBecomeVisible:spec];
 [spec setTarget:root];
 [spec release];
 root.parentController = self;
 
 [self pushController:root];
 [root release];
}
@end

Finally, in your app's delegate, write these code:

-(void)applicationDidFinishLaunching:(UIApplication *)application {
 window.frame = [UIScreen mainScreen].applicationFrame;
...
 MyRootController* rootCtrler = [[MyRootController alloc] initWithTitle:@"Some Title" identifier:@"com.yourcompany.appName"];
 [window addSubview:[rootCtrler contentView]];
}

Why do this? One major reason it that we can now debug the Pref Bundle in Simulator, and speed up development.

iKeyEx 0.1-99c and QuickScroll 0.1-12d released

2009-08-12T17:09:00.003+08:00

iKeyEx 0.1-99c was released, changes were:

Fixed a silly bug causing autocorrection not working for internal input modes.
nlist nows checks for N_ARM_THUMB_DEF just for safety. Probably make it works for 3GS, probably not.
Input modes won't suddenly "reset" now.

QuickScroll 0.1-12d (@r437) fixed a bug where you could scroll horizontally (and actually, vertically too) one pixel too far. Also, it now synchronizes when content size change (e.g. additional content of a web page is loaded, or orientation is changed).

QuickScroll 0.1-12c released (bug fixes)

2009-08-12T03:21:00.003+08:00

QuickScroll 0.1-12c was released. The changes include:

Can scroll to the top in MobileNotes now.
Now the triple tap will iterate all touches, eliminating any possibility that a triple tap (that ought to be caught) is missed.
QuickScroll will be suppressed on non-scrollable views.
QuickScroll (PDF paging) will be suppressed on PDF files with 0 pages (usually those incomplete files).
Japanese localization.

(And the dylib size drained down from 20,000 bytes to 19,968 bytes :( )

0.1-12c was submitted to BigBoss, so it should appear in a few days. Those who can't wait can download from here.

Meanwhile I want to explain one decision. Why triple tap? My initial intension was three-finger touch, but I sometimes hold the device with one hand and the other occupied, so any multitouch ideas are ruled out. But how about, like, triple-tapping the home button? This won't work either, as it is possible that two web views appear on screen simultaneously. This excluded all solutions not using the screen, e.g. using the home/lock buttons, shaking the device, etc.

If you have installed iKeyEx 0.1-99b (make sure you're using 0.1-99b) and experienced Crash and not using 3GS please email me kennytm@gmail.com, preferably with a crash log (and syslog if possible).

QuickScroll 0.1-12b released, supports PDF paging and activating from anywhere

2009-08-11T02:28:00.007+08:00

Download is here, as usual. The seventy people who have installed 0.1-12a should also upgrade for a correct offset calculation in MobileSafari.

Now QuickScroll becomes a 1.5-day project, so there are much more features introduced.

Changes:

Activate QuickScroll from anywhere, like web pages, long text views, tables, MobileTerminal, SpringBoards, just anywhere you can scroll. (Blame MobileSafari for requiring me to introduce this.)

Jump to any page in a PDF file.

Fixed from 0.1-12: Rotating from portrait to landscape position no longer makes the "Close" button unreachable.

Modified: The alert box is now smaller.

Language support for English, Spanish, Italian (thanks Sagitt for a few translations) and Chinese (T+S).

This version should be stable enough and I think no more features are needed (besides more localizations). The very same package shall appear in Cydia repo soon if I receive no bug reports by tomorrow (14:00 at GMT+8).

P.S. The dylib is exactly 20,000 bytes.

QuickScroll 0.1-12 released

2009-08-10T05:20:00.002+08:00

QuickScroll 0.1-12 is released. It's a quick (:p) project done within half a day.

QuickScroll lets you scroll through a Web page (including PDF files and Word documents) quickly. To use:

In a long web page, triple tap on anywhere (hold one finger and triple tap the other to avoid resizing).
A dialog will appear. Drag the green box to scroll around.

iKeyEx 0.1-99b is released (3.0 only)

2009-08-09T20:51:00.006+08:00

Hi girls and guys. 0.1-99f has been released. Maybe check the newest post first before commenting? ;p

iKeyEx 0.1-99b is released. This is a beta version, and the release version will use the version number "0.2".

Compared with the iKeyEx 2.x series, a lot of things are changed that may render an existing keyboard useless.

Firstly, text with traits are not supported yet, meaning keyboard that entirely relying on it such as 1Click@Thai (com.iclick.thai5rowkeyboard) will entirely unusable in this version.

Secondly, libiKeyEx.dylib is rewritten and uses a different API. That means existing keyboard more complex than a standard keyboard, including ℏClipboard and 5 Row QWERTY cannot run. If dependence is not a lot, however, transition to 3.x is pretty easy.

Thirdly, the input mode architecture is changed. Previously, an input mode linked to exactly one layout. It is decoupled on 0.1-99b. For example, I can have a 5-row QWERTY keyboard for English, for Japanese and for Chinese simultaneously. You can do it in the Mix & Match options in Settings → iKeyEx. But because a layout no longer completely defines an input mode, installing a keyboard may not be immediately usable.

Example 1: The Deutsch 5 Row Keyboard (com.ipuhelin.keyboards-5rowdeutsch) will be installed using English (US) as input manager. You can fix it by choosing Settings → iKeyEx → Mix and Match → 5RowDeutsch → Input manager → German (Germany).

Example 2: The Dvorak 5Row Keyboard (hk.alim.dvorak5row) was installed without adding itself to the list of input modes. In 2.x you'd add it from International Keyboards, but this is more complicated on 3.x due to the decoupling. The procedure can be summarized into this pic:

In words,

Go to Settings → iKeyEx
Select "Mix and Match",
then "Create", to create a new input mode.
Now, select "Layout",
and change it to "5-Row DVORAK" at the top.
After that, select "Name"
and give it a name. It must not be empty.
Finally, go back to the iKeyEx screen and tap "Keyboard"
You should be able to see the new input mode. Tap on (+) to add it.

I haven't put in any localization yet. They will appear in the next version.

The 8th property list object

2009-08-07T22:33:00.002+08:00

We've been long told that there're only 7 types of property list objects: CFString, CFNumber, CFBoolean, CFDate, CFData, CFArray, and CFDictionary. These are the only types that can be encoded into a property list without conversion. At least it's what on the surface. There's in fact an eighth type of property list object, called _CFKeyedArchiverUID, to support the NSKeyed[Un]Archiver.

An archived data is actually a binary property list. The encoded objects are laid out linearly, and each object has a UID numbered serially. Objects can be linked to each other, forming an "Object Graph". The links are encoded as _CFKeyedArchiverUIDs.

A _CFKeyedArchiverUID is essentially a 32-bit integer. Why isn't a CFNumber be used? Because it would then be impossible to distinguish between a regular number and a link.

Becuase _CFKeyedArchiverUID is not publicly documented (you can find its trace in the CF source code), it is a perfect type to conceal these supposed-to-be internal information.

But it introduce problems when handling with plist files that have a _CFKeyedArchiverUID. Firstly, Apple's Property List Editor does not recognize them. And even worse, these UIDs are stripped when saved. You can try this: open any .nib file, without any modification save it as another file. You'll find that the new file becomes smaller.

Also, _CFKeyedArchiverUID does not have a specialized -description method. When you NSLog a dictionary containing with object, a <NSCFType: 0xpointer> will be displayed (although CFShow does show the content).

In XML format, _CFKeyedArchiverUID will be output as

<dict> <key>CF$UID</key> <integer>123</integer> </dict>

This means any dictionaries containing a single entry with key CF$UID and an integer value will be converted to a _CFKeyedArchiverUID.

LC_DYLD_INFO_ONLY demystified

2009-08-05T12:09:00.003+08:00

One day after I found no useful info on LC_DYLD_INFO_ONLY, there's already a paste showing the answer (this is Google cache, the original one is already invalidated). I have copied it to pastie in case the Google cache is removed. From the comment we clearly see the intent, and hints to how to decode this command.

Using CPRegularExpression (not!)

2009-08-05T03:11:00.002+08:00

AppSupport.framework contains a class called CPRegularExpression, which performs regular expression matching. Here is an interface and some sample code.

However, I strongly recommend against using this class. Why? Because the backend of CPRegularExpression is in fact regexec. regexec is underpowered and not fast at all (for commonly encountered regexes, at least). Combined with the dynamic nature of Objective-C it means very slow (you can get an IMP to speed this up though). Instead, you should use the more powerful and faster PCRE (it's of course hard to use, but there're plenty of wrappers).

class-dump-z 0.1-11s released.

2009-08-05T02:08:00.003+08:00

class-dump-z 0.1-11s is released. (When iKeyEx 3.0 is released I'll upgrade the versioning to 0.2. Let's hope I won't exhaust all alphabets first.)

Change log:

Won't crash on 3.1 binaries now, but external class info is absent. For example, for SpringBoard, SBAppWindow : UIView will be shown as SBAppWindow : XXUnknownSuperclass.
Private ivar detection. From whether the ivar symbol is exported we can guess if that ivar is @private/@package or @protected/@public. class-dump-z 0.1-11s now recognizes this piece of information.
-h super now works even the method includes non-id/void/SEL types.
Argument reordering now works on Linux (again).

Technical detail on why class-dump-z 0.1-11r crash with 3.1 SpringBoard, and why the external class info cannot be extracted in 0.1-11s.

If you use class-dump-z 0.1-11r on the 3.1 SpringBoard, you'll get an exception saying

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_S_construct NULL not valid
Abort trap

This is because a code similar to this is called:

std::string s = NULL;

Debugging reveals such an error occurs at MachO_File_ObjC::get_superclass_name. What happened is that, the definition of SBAppWindow class of 3.1 SpringBoard resolves to:

 class_t OBJC_CLASS_$_SBAppWindow = {
  isa: OBJC_METACLASS_$_SBAppWindow,
  superclass: NULL,
  cache: NULL,
  vtable: NULL,
  data: (pointer to Data)
};

The superclass pointer is NULL. Which is fine, as such already happens before 3.1. But how can the system know the superclass if the pointer is NULL? Before 3.1, this location is in fact referenced by an entry in the relocation table. At runtime, the dynamic linker will replace data referenced in the relocation table by the real address.

The problem is the 3.1 SpringBoard has no relocation table! We can check:

Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 1
     iextdefsym 1
     nextdefsym 1
      iundefsym 2
      nundefsym 1330
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 1138584
  nindirectsyms 1943
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0

Because of this, class-dump-z 0.1-11r cannot resolve what that address will be, and returns NULL. The code afterward see this NULL unexpected and result in exception.

0.1-11s now checks for this failure, but it's just a bad workaround. Because every externally referred class, like NSObject, UIView, etc. will become XXUnknownSuperclass.

Now, if we don't know the superclass, how can the iPhoneOS be? There, I introduce you the Load Command 0x80000022:

Load command 4
      cmd ?(0x80000022) Unknown load command
  cmdsize 48
00000000 00000000 00106000 00006c6c 00000000 00000000 0010cc6c 00005324 
00111f90 00000198

It is run without the 3.1 SDK. Because this is an unknown command, nm totally fails on the 3.1 SpringBoard.

nm: for architecture armv6 object: SpringBoard malformed object (unknown load command 4)

Googling suggests it appears only on 3.1+ and Snow Leopard+. That means the ABI will be changed on SL? And relocation table won't be used? Maybe. With a 3.1 SDK, we can see that the load command is in fact called LC_DYLD_INFO_ONLY. The content is


struct dyld_info_only_32 {
  uint32_t rebase_off;
  uint32_t rebase_size;
  uint32_t bind_off; // 0x106000
  uint32_t bind_size; // 0x6c6c
  uint32_t weak_bind_off;
  uint32_t weak_bind_size;
  uint32_t lazy_bind_off; // 0x10cc6c
  uint32_t lazy_bind_size; // 0x5324
  uint32_t export_off; // 0x111f90
  uint32_t export_size; // 0x198
}

This load command is not documented on Apple's site either, and google yields no useful result at all. But we can find the string _OBJC_CLASS_$_UIWindow at 0x106ea0 inside the so called "bind" table, so definitely something important can be found here.

Partial support for iKeyEx layout.plist on 3.0

2009-08-03T02:46:00.006+08:00

The layout.plist to UIKBKeyboard conversion function is done. As promised, an executable that converts layout.plist to .keyboard files can be downloaded from here.

This program converts a layout.plist into 8 .keyboard files which can be loaded by UIKit natively. For example, if you want to install the Colemak keyboard,

ssh into your device

Download layout-plist-to-keyboards by typing:

wget http://networkpx.googlecode.com/files/layout-plist-to-keyboards

Change that file to an executable:
```
chmod a+x layout-plist-to-keyboards
```

Download the layout.plist for Colemak:

wget http://networkpx.googlecode.com/svn/trunk/hk.kennytm.Colemak/deb/Library/iKeyEx/Keyboards/Colemak.keyboard/layout.plist

Perform the conversion:

./layout-plist-to-keyboards layout.plist

Check that there are 8 .keyboard files in the directory.
```
ls
```
Move all these keyboards to UIKit.framework (requires root permission):
```
mv *.keyboard /System/Library/Frameworks/UIKit.framework/
```

Now all QWERTY keyboards are replaced by Colemak.

If you don't want to replace the QWERTY keyboard, you can rename them to something else. Say, you'll never use the Thai keyboard, then name them iPhone-Portrait-Thai.keyboard, etc. Now you can activate the Thai keyboard to use the custom one.

This solution is not a replacement of a full iKeyEx. For example, 5-Row QWERTY won't work completely because the custom-code injecting part is not done yet (but Punctuation Mod should work). hClipboard won't work at all since it's no a layout.plist. Other caveats are listed here. Please don't host Cydia packages publicly that uses this solution, because every keyboard will become mutually exclusive now :) Only for personal use.

class-dump-z 0.1-11r released, supports Windows

2009-08-01T22:29:00.003+08:00

A minor revision of class-dump-z was released. The new version:

Adds a Windows x86 version.
Arguments can be placed arbitrarily (e.g. class-dump-z UIKit -a -A now works).

Updated on Aug 2:

If you have installed the iPhone SDK, the -y switch is no longer necessary when using -h super.
Also, using -h super won't crash now.

GriP 0.1-11r Released to fix Crashing with Exchange mail accounts

2009-08-01T11:33:00.004+08:00

As titled. If you are experience crashes when new mails arrive to an Exchange mail account, please upgrade to GriP 0.1-11r. Detail can be found in issue 242.

GriP, GriPGRIP & class-dump-z 0.1-11q Released

2009-07-31T22:00:00.003+08:00

Starting from this version, Memory Watcher will not be included in GriP. Instead, it will be moved to a new package, "GriP Really Impressive Pack" (GriPGRIP). The distinction is that, GriP shall only contains the most needed extensions, i.e.

Mail
Push notification
SMS (by xshad0w, not included in 0.1-11q yet),

the 2 themes, Default and Double Height Status Bar (also by xshad0w, also not included yet), and 3 modal table themes, Default, HUD and Translucent Black (I may remove the last one too). On the other hand, GriPGRIP is a collection of all other stuff that will utilize GriP, like calendar events, remote Growl notifications, themes, etc. (The distinction is because GriP is growing to 1 MiB pretty soon. I want to keep the size down.)

Anyway, 0.1-11q is mainly an issue-fixing release, that include:

Issue 223. You can configure some GriP messages to wake the device up.
Issue 176. An new option so that GriP message can be expanded by default.
Issue 209. You've Got Mail will show the user name of the mail owner, instead of the address.
Issue 168. All apps can be selected shown in Game Mode. The UI is improved too :)
Issue 172. Multiple push notifications from the same app will be coalesced.
Suspension behavior can be updated properly.

If you have GriP 0.1-11p, you don't need GriPGRIP 0.1-11q, because there's nothing new.

And class-dump-z. There are 4 new features in this release:

The -N flag, which allows you to disable struct name prettifying (__CFArray* → CFArrayRef).
Optional properties. Such a thing does not exist directly, but can be inferred when its getter and setter are optional. (See the UITextTraits protocol for example).
The -h proto flag, which will hide all methods that's just adopting a protocol.

-h super

Note that if you use the -h super flag directly on the Mac, it is almost certain that class-dump-z will crash. This is because to get the superclass info class-dump-z must open some framework bundles, and by default it will search in /System/Library/..., which certainly does not contain anything iPhoneOS-related. You have to provide the -y flag to specify the sysroot as well. For those who have installed SDK, you need to add

-y /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk

Text input on 3.0, Part 4 — UIKBKeylistReference

2009-07-30T14:14:00.002+08:00

When I was preparing this whole stuff many wrong directions were taken and generates quite a number of utilities that could be useful later. Who knows? Head to http://code.google.com/p/networkpx/source/browse/etc if you are interested.
You will need UIKB*.h to compile my examples. The headers can be found in my other project page at github. However, besides ActorKit no header there is actually complete (yet).

Hopefully it's the last of the series. I'm now making a convert from layout.plist to .keyboard files. This is a major foundation for backward-compatibility with previous keyboards, and also allows you to use custom keyboards on 3.0 without iKeyEx (but, without all the benefits like unlimited numbers of keyboards). The basic idea is described in Part 1.

~~Anyway...~~

Unlike the old UIKeyboardLayoutRoman, the new UIKeyboardLayoutStar is "scalable", that means this keyboard will still look sharp even if used on a 10" tablet PC. This is because the whole UIKeyboardLayoutStar is drawn at run time with Rounded rectangles and Gradients and Shadows and stuff. It's vector graphics. (In constrast, UIKeyboardLayoutRoman is precomposed into a PNG image.)

But even the layout has the potential to be scalable, all keyboards I've created before (using Star) is not, because I haven't told it what to scale. It's like an iPhone app, most of them can be easily presented in landscape mode, but if the programmer doesn't implement the -shouldAutorotateToInterfaceOrientation: method the app will always stuck in portrait mode.

As I've repeated many times, keyboard construction occupies 70% mass of UIKit, and that's roughly 8 MiB of code. There are 148 keyboards defined in UIKit so far, so each keyboard will be defined by 55 KiB of code and data. And unfortunately, most of these are code. Now, ARM's register-relative addressing mode allows a maximum span of 11 bits, i.e. 8 KiB, but there's 55 KiB of code! That means each single function must be chopped into chunks by the compiler. To the analyzing decompiler (ravel-arm), it is a difficult situation, because an unconditional branch at the end may mean a tail-function-call. The data after may be interpreted as code, and ruins all stored computation in between. This is happening in the keyboard-defining functions, so by static analyzing the code I can only get to how Apple defines keys up to the Capital Letter U (that's Q, W, E, R, T, Y, U), and then a branch across some mis-codified data, and all selector and class info are lost.

So? Initially I dumped all 148 keyboards and analyzed the resulting archive, but this doesn't tell us the implementation detail. Eventually I hooked objc_msgSend to give what's being called (since I knew only Obj-C functions are called and no structs are returned, I've ignored all other messaging variants).

The result is over 9000 bytes and I can't pastie it. But I can pastie part of the decompiled result. The UIKBKeylistReference part is what I was missing. Now let's plug in the keyboard, and we get... a dangling Q as we've expected.

But if we resize the keyboard from 320x216 to 240x216? Yes, the key is correctly scaled down:

And removing the references even make the dimensions wrong, so the scaling is really the reference in effect.

(When the reference is defined, the original geometry of the keys will be overridden).
(Also note that this keyboard cannot be scaled vertically, because the vertical measures of the rows are absolute values (pixels).)

One puzzling property of references is the "flag". Yes we know it must be bitfield of some sort, but what? If we decompile the class UIKBKeylistReference we get:

1 = isKeysetReference
2 = isKeylistReference
4 = isKeysReference
8 = isNamedKeyReference
0x10 = isKeyIndexReference
0x20 = isKeyIndexRangeReference
0x40 = isGeometryReference
0x80 = isAttributesReference

Apparently it refers to while the name elements interpreted.

Why we care about scaling? One, it allows us to define landscape and portrait keyboards with the same content (the keys), and to switch between the two we just need to change the outermost dimension, and the rest will be seamlessly changed. And two, who said the keyboard size must be fixed at 320x216 / 480x162? What if there is really an Apple tablet and uses the iPhoneOS? Or there's an iPhone mini? The resolution can't be the same, so the keyboard size must be altered as well. On 2.x we're forced to hard-code these 4 numbers because UIKeyboardLayoutRoman expects us to do so. Now having the better solution with UIKeyboardLayoutStar, I'd rather spend more time to take advantage of it and make iKeyEx more robust in the future.

Multiple Row Selection with UITableView — The Easy Way

2009-07-27T17:20:00.004+08:00

Cocoa with Love introduced the hard way to implement multiple row selection for 2.x firmware. On 3.x, there is a much easier way to do the same, if you dare you embrace undocumented methods.

Multi-selection is regarded as one of the editing style. Therefore, to make a cell multi-selectable, implement this in your UITableViewDelegate:

-(UITableViewCellEditingStyle)tableView:(UITableView*)tableView editingStyleForRowAtIndexPath:(NSIndexPath*)indexPath {
  ...
  return 3;
}

The "3" here means multiselection. The result is like this:

To get the selected rows, call the -indexPathsForSelectedRows method on the table view.

NSArray* selectedRows = [tableView indexPathsForSelectedRows];

If you dislike the red check mark, you can use the undocumented multiselectCheckmarkColor property to change it. Unfortunately, it has to be applied on the whole table.

tableView.multiselectCheckmarkColor = [UIColor blueColor];

The light blue background color cannot be changed unless you subclass or categorize UITableViewCell and override the -_multiselectBackgroundColor method, like this:

-(UIColor*)_multiselectBackgroundColor { return [UIColor yellowColor]; }

class-dump-z & GriP 0.1-11p released

2009-07-18T06:14:00.003+08:00

Downloads available in http://code.google.com/p/networkpx/downloads/list as usual.

What's new in GriP: Again, another attempt to fix issue 129.

What's new in class-dump-z:

Doesn't crash with WebKit.framework anymore.
- The offending struct is XXStruct_GIci6C. Due to some strange thing in the GCC that compiles WebKit, the type of function pointer is encoded into "^" (pointer to !#$^*@#NO_CARRIER) instead of "^?" (pointer to some unknown stuff). The original class-dump won't handle this stuff either, and spit out a huge error string.
Doesn't crash with Symbolication.framework anymore.
- The offending class is VMUCallTreeNode, which contains a union to a pointer to a static-typed objective-C class, which will cause a segfault in 0.1-11o. This was just an error in class-dump-z.
Forward declaration of some structs were missing if it is nested. Not anymore.
Fixed issue 187. It was because class-dump-z missed the LC_LOAD_WEAK_DYLIB commands.
Fixed issue 198.
Linux version: PCRE is static-linked. No need to search for the right version of libpcre.so anymore.

Analyzing Objective-C's for-in loop (fast enumeration)

2009-07-09T02:53:00.002+08:00

Objective-C 2.0 supports for-in loop like this:

for (XXSomeClass* obj in someArray) {
  [obj doSomething];
}

it's nice to the programmer, but painful for the disassemblers because the simple loop above is translated into this bulky ASM:

            ldr               r1,[pc,#0xe8]
            mov               r3,#0x0
            str               r3,[sp,#0x44] // state
            str               r3,[sp,#0x48] // itemsPtr
            str               r3,[sp,#0x4c] // mutationsPtr
            str               r3,[sp,#0x50] // extra[0]
            str               r3,[sp,#0x54] // extra[1]
            str               r3,[sp,#0x58] // extra[2]
            str               r3,[sp,#0x5c] // extra[3]
            str               r3,[sp,#0x60] // extra[4]
            ldr               sl,[pc,r1]
            add               r3,r3,#0x10
            str               r3,[sp]
            mov               r1,sl
            add               r2,sp,#0x44
            add               r3,sp,#0x4
            bl                objc_msgSend (stub)               ; [someArray countByEnumeratingWithState: objects:count:]
            subs              r5,r0,#0x0
            beq               loc_000100
            ldr               r3,[sp,#0x4c]
            ldr               r3,[r3]
            mov               r6,r3
loc_000094: mov               r4,#0x0
            b                 loc_0000a4
loc_00009c: ldr               r3,[sp,#0x4c]
            ldr               r3,[r3]
loc_0000a4: cmp               r6,r3
            beq               loc_0000b4
            mov               r0,r8
            bl                objc_enumerationMutation (stub)
loc_0000b4: nop               
            nop               
            nop               
            nop               
            nop               
            add               r4,r4,#0x1
            cmp               r5,r4
            bhi               loc_00009c
            mov               r3,#0x10
            str               r3,[sp]
            mov               r0,r8
            mov               r1,sl                             ; "countByEnumeratingWithState:objects:count:"
            add               r2,sp,#0x44
            add               r3,sp,#0x4
            bl                objc_msgSend (stub)               ; [? ?]
            subs              r5,r0,#0x0
            ldrne             r3,[sp,#0x4c]
            ldrne             r3,[r3]
            bne               loc_000094
loc_000100: ...

The disassembled code is

NSFastEnumerationState enumState;
enumState.state = 0;
enumState.itemsPtr = NULL;
enumState.mutationsPtr = 0;
enumState.extra[0] = 0;
enumState.extra[1] = 0;
enumState.extra[2] = 0;
enumState.extra[3] = 0;
enumState.extra[4] = 0;
NSUInteger count = 16;
id buffer[10];
do {
 NSUInteger copiedItemsCount = [someArray countByEnumeratingWithState:&enumState objects:buffer count:count];
 for (unsigned long i = 0; i < copiedItemsCount; ++ i) {
  if (*(enumState.mutationsPtr) != 0)
   objc_enumerationMutation(someArray);
  [enumState.itemsPtr[i] doSomething];
 }
} while(copiedItemsCount != 0);

What does the state and extra do? Up to the implementor. (And objc_enumerationMutation simply kills the program and reports that a mutation happened.)

Text input on 3.0, Part 3 -- Hooking

2009-07-08T02:40:00.003+08:00

I think the best solution for now is put a hook at -[UIKeyboardLayoutStar setKeyboardName:appearance:]. If we see a custom name, we could put our own keyboard into the cache dictionary.

Who called -[UIKeyboardLayoutStar setKeyboardName:appearance:]? From the backtrace,

#1  0x30b16dae in -[UIKeyboardLayoutStar showKeyboardType:appearance:orientation:] ()
#2  0x30a1da78 in -[UIKeyboardImpl updateLayout] ()
#3  0x30a1a175 in -[UIKeyboardImpl setDelegate:force:] ()
#4  0x30a1815a in -[UIKeyboardImpl setDelegate:] ()

and the calling code of -[UIKeyboardLayoutStar showKeyboardType:appearance:orientation:] is

[self setKeyboardName:UIKeyboardGetKBStarKeyboardName(
                       UIKeyboardGetCurrentInputMode(),
                       orientation, keyboardType)
           appearance:appearance];

and UIKeyboardGetKBStarKeyboardName is hard-coded to generate the expected keyboard name. But before we start we must ensure the input mode can be handled by UIKit! In 3.0 there's a function UIKeyboardGetSupportedInputModes() which returns the 46 hard-coded input mode names. Well, we could of course use MSHookFunction to replace it, but I'd like to see if it's possible to do it with Objective-C only, because MSHookFunction is not available on x86. And actually there is a trick — we see that UIKeyboardGetSupportedInputModes() returns a shared object, which is an NSMutableArray*. What does this mean? Well, we can call this function once on initialization, modify it, and from that point on UIKeyboardGetSupportedInputModes() will record our change.

Then we need to trick UIKit into thinking our keyboard uses UIKeyboardLayoutStar. This information is extracted in UIKeyboardInputModeUsesKBStar(), which is again hard-coded. However, this function uses an NSDictionary* which cannot be accessed outside. So we can't test on simulator? Not really, we could actually hook -[NSDictionary initWithObjectsAndKeys:] instead, and swap it back to the original method when done.

There are a few other hard-coded dictionary in UIKeyboardInputManagerClassForInputMode(), UIKeyboardLayoutDefaultTypeForInputModeIsASCIICapable(), UIKeyboardGetKBStarKeyboardName(), etc. After all these are replaced we can get a totally empty keyboard without error — a good start.

Now we can insert the actual hook to -setKeyboardName:appearance: and set the keyboard. This part is not difficult because there's are reference implementations, like that disassembly of getKeyboardiPhonePortraitQWERTY() and dump of various keyboard files.

Text input on 3.0, Part 2.

2009-07-06T20:15:00.002+08:00

In the last part, we found that we may be able to use the .keyboard format via GetKeyboardDataFromBundle(). This function is only called from -[UIKeyboardLayoutStar setKeyboardName:appearance:], and is not exported, so it seems .keyboard really only contains the keyboard geometry.

To see if it really works, let's save an internal keyboard into a file:

[NSKeyedArchiver archiveRootObject:UIKBGetKeyboardByName(@"iPhone-Portrait-Arabic") toFile:@"/tmp/iPhone-Portrait-Arabic.keyboard"]

unfortunately, UIKBGetKeyboardByName is not an exported symbol, so we have to use nlist or lookup_function_pointers. Now, rename this file to e.g. iPhone-Portrait-QWERTY.keyboard (the English keybaord) and place it to UIKit.framework — and, indeed, the English keyboard becomes Arabic!

But how to create a UIKBKeyboard? A UIKBKeyboard has 3 major components: name (NSString*), visualStyle (NSString*) and keyplanes (NSArray*). The name is just the name you'd expect (iPhone-Portrait-Arabic), while the visualStyle must be one of @"iPhone-Standard" or @"iPhone-Alert". These are easy to construct. The meat are all in the keyplanes variable. The keyplanes is an array of UIKBKeyplane*. Difference keyplanes are switched between using the shift and 123/ABC keys. Again, each keyplane has a name (NSString*, e.g. @"Capital-Letters"); keylayouts, an array of UIKBKeylayout*; attributes, a UIKBAttributeList* and finally supported-types, an array of NSString*, etc., etc...

The content is summarized into http://code.google.com/p/networkpx/wiki/keyboard_file_spec. With the format analyzed, we can try to create our own keyboard using this new format.

Text input on 3.0, Part 1.

2009-07-06T03:56:00.002+08:00

2 little notes:

GriP 0.1-11o has been released to fix issue 129, and that's the only change.
class-dump-z 0.1-11o has been released for Mac, iPhone and Linux. See this page for a comparison of other class-dumps.

The rest of this text is written at the same time I'm doing the reverse engineering. You may feel confused, because that's how researching goes.

The text input framework is greatly revamped in 3.0. The new UIKit, which is 5 times bigger than 2.0, has 70% of weight taken up by keyboard layouts. And the nontrivial letter-based input managers, once part of UIKit, is moved into its private framework, TextInput.

What was changed? As the first try, let's view the view hierarchy, although this time we can do it in compile time. From the class-dump we know the old UIKeyboard class is still there, so let's run the statement
UILogViewHierarchy( [UIKeyboard activeKeyboard] );
when the keyboard is present.

So we got this with the English keyboard:


<UIKeyboard: 0xd47980; frame = (0 264; 320 216); opaque = NO; layer = <CALayer: 0xd47d00>> ({{0, 264}, {320, 216}}, (null), #0, 3)
..<UIKeyboardImpl: 0xd3da20; frame = (0 0; 320 216); opaque = NO; layer = <CALayer: 0xd37b40>> ({{0, 0}, {320, 216}}, (null), #0, 3)
....<UIKeyboardLayoutStar: 0xd48650; frame = (0 0; 320 216); layer = <CALayer: 0xd4ba40>> ({{0, 0}, {320, 216}}, (null), #0, 3)
......<UIKBKeyplaneView: 0xd59250; frame = (0 0; 320 216); userInteractionEnabled = NO; layer = <CALayer: 0xd59300>> ({{0, 0}, {320, 216}}, (null), #0, 3)
........<UIKBKeyView: 0xd595e0; frame = (1 119; 40 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd596b0>> ({{1, 119}, {40, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59af0; frame = (279 119; 40 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59b60>> ({{279, 119}, {40, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59d10; frame = (1 173; 38 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59d50>> ({{1, 173}, {38, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd59e20; frame = (41 173; 38 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd59e60>> ({{41, 173}, {38, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd4dca0; frame = (81 173; 158 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd4dab0>> ({{81, 173}, {158, 42}}, (null), #0, 4)
........<UIKBKeyView: 0xd5bd10; frame = (241 173; 78 42); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0xd5bd50>> ({{241, 173}, {78, 42}}, (null), #0, 4)

In fact, all other keyboards — except the handwriting ones, and emoji also — now uses UIKeyboardLayoutStar to manager the layout. (The old UIKeyboardLayoutRoman is still there.)

The 6 UIKBKeyView are the special keys, as shown here:

The class dump of UIKeyboardLayoutStar shows some interesting members, e.g. m_keyboardName, m_keyplaneName, the list of dictionaries, etc. Take the Arabic keyboard as example.

m_keyboardName = @"iPhone-Portrait-Arabic"
m_keyplaneName = @"Letters"
m_states is a dictionary of 35 entries, with keys as UIKBKey to values as NSNumber (integers).
m_keyboard is the keyboard (UIKBKeyboard), which contains a lot of geometry info.

If we can replace UIKBKeyboard, then we can make our own layout. Where is that UIKBKeyboard set? Looking at the disassembly of UIKit (which is kinda painful now due to its huge size), we see the relevant part is at -[UIKeyboardLayoutStar setKeyboardName:appearance:]:

NSString* UIKeyboardGetCurrentInputMode();
NSBundle* UIKeyboardBundleForInputMode(NSString* mode);
NSData* GetKeyboardDataFromBundle(NSString* name, NSBundle* kbbundle);
NSBundle* _UIKitBundle();
UIKBKeyboard* UIKBGetKeyboardByName(NSString* name);

@implementation UIKeyboardLayoutStar
-(void)setKeyboardName:(NSString*)name appearance:(UIKeyboardAppearance)appr {
 UIKBKeyboard* keyboard = [m_keyboards objectForKey:name];
 if (keyboard == nil) {
  NSData* keyboardData = GetKeyboardDataFromBundle(name, UIKeyboardBundleForInputMode(UIKeyboardGetCurrentInputMode()));
  if (keyboardData == nil)
   keyboardData = GetKeyboardDataFromBundle(name, _UIKitBundle());
  if (keyboardData != nil) {
   NSKeyedUnarchiver* keyboardArchive = [[NSKeyedUnarchiver alloc] initForReadingWithData:keyboardData];
   keyboard = [keyboardArchive decodeObjectForKey:@"keyboard"];
   [keyboardArchive finishDecoding];
   [keyboardArchive release];
  } else
   keyboard = UIKBGetKeyboardByName(name);
 }
 
 if (keyboard == nil)
  return;
 
 // the rest are not interesting at the moment.
}
@end

So they defined 3 ways to fetch the UIKBKeyboard, (1) GetKeyboardDataFromBundle from the keyboard bundle (/System/Library/TextInput/TextInput_xx.bundle/), (2) GetKeyboardDataFromBundle from the UIKit bundle, and (3) UIKBGetKeyboardByName.

What does GetKeyboardDataFromBundle do? Basically,

GetKeyboardDataFromBundle (name, bundle) {
  Go to the bundle's resource path;
  Open ".keyboard";
  Return the content of this file as NSData*.
}

But there isn't a single .keyboard file in the TextInput bundles! How about in UIKit.bundle? No, not even one. So the only path taken would be by UIKBGetKeyboardByName. And what does it do? Search from a function table with that name, and call that function. This probably explain the 70% mass of UIKit — these are functions to define the keyboards. Why they don't use the .keyboard files is a mystery, but from this it seems we can exploit the format to let iKeyEx run on it.

GriP 0.1-11n Released

2009-06-24T02:27:00.003+08:00

GriP 0.1-11n (r344) is released and can be downloaded from http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.grip-0.1-11n.deb. This version will be the last revision with Memory Watcher bundled. Starting from next revision, Memory Watcher will be moved to another package.

GriP 0.1-11n features following changes:

(r327) GriP will use Preference Loader to inject the preferences. You'll have to install PreferenceLoader first.
(r327) GriP Modal Tables can now be themed.
(r331) All recent GriP messages will be logged.
(r332) The detail disclosure thing (▼) is available in You've Got Mail again.
(r333) Game mode now works in 3.0
(r334) Dismissing the modal table by home button works correctly in 3.0.
(r340) Added Polish localization.
(r343) 3.0 Push notification plug-in.
(As usual) Other bug fixes.

Note that if you've downloaded the r341 version that I've uploaded by mistake this morning, you'll experience a memory leak. Please upgrade immediately.

Sending push notification locally.

2009-06-23T21:36:00.003+08:00

So far I haven't found a reliable push notification app yet to test GriP Push Notification (QuickPigeon is the most reliable one I've ever tried, but it held up 2 trial mails quite a long time).

So what about faking push notifications? After a push notification is just a TCP packet. The first thing I do is to set a breakpoint on the most suspicious method -- -[SBRemoteNotificationServer connection:didReceiveMessageForTopic:userInfo:] (0x9ffa5). And indeed, when a push notification comes, the break point is hit (multiple times).

I used the breakpoint command

call (void)CFShow($r2)
call (void)CFShow($r3)
call (void)CFShow(*$sp)
c

to record the arguments. And not surprisingly, the userInfo is an NSDictionary of the payload, like

{
  aps = {
    sound = "hello.aif";
    alert = "Hello world!";
  };
};

The Apple's documentation on this is more in detail. The "topic" parameter is the bundle ID of the app to receive the notification, and connection is an APSConnection object.

Given these results, we can craft our notification packet with a Mobile Substrate extension.

Right now, you can download FakePushNotification.zip and test it out. To install:

Put the .dylib and the .plist to /Library/MobileSubstrate/DynamicLibraries
Put the executable (FakePushNotification) to /usr/bin and chmod a+x it.
Respring

In command line, type e.g.:

FakePushNotification com.yourcompany.pushEnabledApp -
aps={
  alert={
    body = "Hello world!";
    action-loc-key = "Welcome!";
  };
};
^D

The "payload" must be in plist format, not JSON format.

Note: the program is written to demonstrate how it can be done. It is not bug free. In particular, sending a string that cannot be interpreted as a plist will cause your device crash and go into safe mode. Use with caution.

openURL for 3.0 (and 2.x too, probably)

2009-06-22T20:33:00.003+08:00

openURL is part of Erica Utilities. On 3.0, it is not usable anymore because Apple now always throw a "There can only be one UIApplication instance." error even if [UIApplication sharedApplication] still returns nil.

But openURL doesn't require UIApplication. The core method, -[UIApplication openURL:] is actually a wrapper of the undocumented function, GSEventSendApplicationOpenURL.

I've rewritten openURL using GSEventSendApplicationOpenURL, and can be downloaded from http://code.google.com/p/networkpx/downloads/detail?name=openURL.zip. Note that GSEventSendApplicationOpenURL relies on the Graphics Services private framework, and this framework isn't very stable, so there is a chance it won't work on 4.0.

You'll notice that there's now a new argument, "port-name". That's actually not quite useful, but I found that it's used in UIKit, so just added it in. (One more reason is that, the function that obtains the default port is renamed in 3.0.) And that piece of UIKit code doesn't work :(.

3.0's presentModalViewController:withTransition:

2009-06-22T03:37:00.002+08:00

In 3.0 the algorithm -presentModalViewController:animated: is changed. The most prominent change I can see is that the modal view controller will always use the application frame instead of the parent view controller's frame. This breaks modal table theming in GriP. Fortunately, probably Apple also thinks the change to too drastic, they've also provided -_legacyPresentModalViewController:withTransition: that uses the old algorithm, so until iPhoneOS 4.0 comes we can still ignore this problem.

The 3.0's -presentModalViewController:animated: calls the undocumented method -presentModalViewController:withTransition: as a backend, where the 2nd parameter "transition" is an integer. For official SDK there're 5 official transitions (present and dismiss count as 2): Slide up/down, Flip to left/right, and cross fade.

But -presentModalViewController:withTransition: supports a bit more:

Parameter	Transition
0	None
1	Push from right to left
2	Push from left to right
3	Push from bottom to top
6	Fade
7	Push from top to bottom
8	Slide from bottom to top
9	Reveal from top to bottom
10	Flip from left to right
11	Flip from right to left

3.0 Status

2009-06-20T15:58:00.005+08:00

Updates:

iKeyEx beta for 3.0 has been out there for weeks. Check the newest posts.
Calculator.searchBundle won't work until someone completely rewrites Spotlight. The guy behind iFile tried to enable custom search bundles before, but the overly-optimized interface made it impossible without rewrites.

iPhoneOS 3.0 has been released and jailbroken, and I've also upgraded, so from now on my primary focus of OS will be 3.0, instead of 2.2.1.

GriP worked pretty well on 3.0. It is predictable as GriP was designed to be as SDK-compatible as much, and I've tested it on 3.0b1 (simulator) from the start.

But iKeyEx is totally different. It cannot work on 3.0 without in-depth investigation on UIKit again. The primary reason is in 3.0 Apple redesigned a lot in language UI. 80% code of UIKit is about the new keyboard UIKeyboardLayoutStar. The input manager is so heavy that split into its own private framework, TextInput.framework. That carries away UIKeyboardInputManagerAlphabet which iKeyEx relies on.

iKeyEx will be 3.0-ready, of course, but it will be a long time to wait.

(And because 3.0 supports single-clipboard copy-and-paste natively, I'll discontinue the current form ℏClipboard. I may write a multi-clipboard extension though.)

As for the side-projects, Calculator.searchBundle does not work as I expected. I need to check the source code of Search.framework again. And I haven't checked if AdKiller works or not.

An incomplete list of entitlement keys

2009-06-07T19:59:00.003+08:00

Key	Type	Used by	Checked by	Notes
com.apple.springboard.debugapplications	boolean	gdb gdbserver	SpringBoard: _SBXXCancelWatchdogAssertionForProcess _SBXXRenewWatchdogAssertionForProcess _SBXXAddWatchdogAssertionForProcess _SBXXLaunchApplicationForDebugging
com.apple.springboard.launchapplications	boolean	iapd Cydia.app Preferences.app Web.app	SpringBoard: _SBXXLaunchApplicationWithIdentifier
com.apple.springboard.opensensitiveurl	boolean	CoreLocation.framework dataaccessd ManagedConfiguration.framework AppStore.app MobileAddressBook.app MobileMail.app MobileMusicPlayer.app MobileSafari.app MobileStore.app Preferences.app locationd	SpringBoard: _SBXXOpenSensitiveURL
com.apple.springboard.wipedevice	boolean	dataaccessd Preferences.app	SpringBoard: _SBXXDataReset
com.apple.springboard.setnowplayinginformation	boolean	YouTube.framework MobileMail.app MobileSafari.app YouTube.app	?
com.apple.springboard.activateawayviewplugins	boolean		SpringBoard: _SBXXEnableLockScreenBundle
com.apple.remotenotification.server.preferences	boolean	Preferences.app	SpringBoard: _SBRNSetBundleIdentifierTypes
get-task-allow	boolean	gdb gdbserver	?	The only documented entitlement key, means "Can be debugged".
task_for_pid-allow	boolean	ReportCrash ps gdb gdbserver	kernel
keychain-access-groups	array of strings	Too many	?	Everyone uses it has the value (apple).
application-identifier	string	Too many	?	The content may not be the same as the bundle ID. For example, the application-identifier for AppStore apps will be (10 random characters).(bundle ID)
allow-obliterate-device	boolean	SpringBoard	?
seatbelt-profiles	array of strings	MobileMail.app MobileSafari.app Web.app	kernel?
modify-anchor-certificates	boolean	Preferences.app	?

The entitlement of a task can be obtained using the undocumented SecTask*** functions. Because of this, a library can define a set of entitlement keys other applications using it must follow.

GriP 0.1-11m released

2009-06-05T03:53:00.002+08:00

GriP 0.1-11m (r323) is released, and can be downloaded at http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.grip-0.1-11m.deb.

The biggest change in 11m is the support for "Modal Table View". You've Got Mail has been modified to support this. The new mail message now looks like:

Note that the "detail disclosure button" is not there. When you tap on the message a table will pop up:

You can click on each item to view the message (not as ideal as MobileMail.app, but still readable):

Why is this change? My rationale is that, if you want to show the detail, you are focusing on the message, and stuffing that much information in that message is not good. If you're focusing on it, why not maximize the view? Hence it is shown as a "Modal Table View".

Does this contradict with the philosophy behind GriP (unobstructive)? No, because this modal thing already has user confirmation, while for alert boxes they just pop up with no prior consent.

(If you don't like this, you can always downgrade to 0.1-11k available in http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.grip-0.1-11k.deb.)

Displaying UIView hierarchy of a compiled app

2009-06-04T23:50:00.005+08:00

Suppose you want to investigate the view hierarchy of some app you did not write. There is a simple way using GDB:

Compile UIUtilities.m into a dylib or bundle. You can download it precompiled here.
Upload the file to the device, e.g. /var/mobile/UIUtilities.bundle. Do not put it in /var/root/.
Now attach gdb to the executable you want to investigate.
Inject UIUtilities to the executable. In the gdb prompt, type:
call (int)dlopen("/var/mobile/UIUtilities.bundle", 10)
If a nonzero value appears, injection succeeded.
Finally, type:
call (void)UILogViewHierarchy((int)[[UIApplication sharedApplication] keyWindow])
The UIView hierarchy of the key window will be recorded in syslog.

Testing various Regular Expression solutions, Part 1.5

2009-06-01T18:56:00.003+08:00

One thing I really like about blogs is that you can exchange ideas and learn something you naturally won't know for a lifetime because you are not an expert. (And I hate you, Xanga Friends Lock users.)

Firstly, I have added the 3 options -DNS_BLOCK_ASSERTIONS=1 -DRKL_CACHE_SIZE=307 -DRKL_FAST_MUTABLE_CHECK=1 as instructed and able to reduce the running time from 1.7s to 0.7s, but still quite slow compared with ICU at 0.25s. Then I increased the cache size to 709, and the running time is now 0.4s. Further increasing to 1619 starts make the running time longer probably due to excessive memory use.

RKL,307 RKL,709
-------------------
0.691226s 0.384044s
0.692440s 0.381194s
0.684274s 0.381550s
0.689568s 0.382489s
0.683985s 0.384499s

Then I consider rolling all 16 pattern into one big RegExp like RE1|RE2|RE3|.... One problem with this is each pattern must not have back-references, but this is fine for wildcard rules since back-references do not appear at all. But another nasty thing is that AdBlock Plus rules has the so called filter options which prevents simple merging to take place. Anyway, the running time after merging are:

ICU       PCRE      regex.h   RKLite    JSRegExp
-------------------------------------------------------------
0.260404s 0.189028s 2.526485s 0.268121s 0.141875s AllInOne
0.256532s 0.160290s 2.526872s 0.264368s 0.141586s
0.257263s 0.160135s 2.536937s 0.266934s 0.142434s
0.256900s 0.160222s 2.532003s 0.265274s 0.141467s
0.256744s 0.160232s 2.528987s 0.264851s 0.142299s
-------------------------------------------------------------
0.253102s 0.067551s 0.235714s 0.382489s 0.113274s SeveralInst (median of 5)

So while RegexKitLite does become faster, all other methods are dragged down, esp. the POSIX ERE. This confirms johne's statement.

There are some other developments: (1) I've found the fnmatch function that does wildcard matching, but the running time is 0.8s, probably due to the use of recursion and implicit calls on locales. (2) Then I've written a specific routine that checks using pure C string, and the result is dramatic -- the function only takes 0.02 seconds! This restored some of my logical beliefs :D.

fnmatch5
---------
0.021076s
0.021167s
0.021065s
0.021229s
0.021238s

The new source code can be found in: http://pastie.org/496783.

Testing various Regular Expression solutions, Part 1

2009-05-31T21:40:00.004+08:00

Edit: Please check also Part 1.5 for some further analysis.

For AdKiller to work properly, Regular expression matching should be supported. There are various RegExp solutions for the iPhoneOS:

regex.h
ICU's regular expression engine
PCRE
RegexKitLite, which is an ObjC wrapper of the ICU engine.
JavaScriptCore's regular expression engine.
Do it yourself!

For Ad blocking, we only need to test if the Regular Expression matches, so I've limited the test case to a simple RegExp.test only.

In this part, I'll test for the AdBlockPlus "wildcard rules", where a * matches anything, and optional | at the beginning or end to indicate an anchor. Because of the simplicity, we can actually do the test without using RegExp (e.g. using CFStringFind).

From the complexity of the RegExp engines, we would expect the time taken follow this order:

CFStringFind << regex.h << JSRegExp < ICU < RegexKitLite < PCRE

But somewhat surprisingly, the actual time taken is like this:

ICU       PCRE      regex.h   RKLite    JSRegExp  CFStrFind
-----------------------------------------------------------
0.253456s 0.069918s 1.726371s 1.651787s 0.139437s 0.078097s
0.275680s 0.067150s 1.715131s 1.652074s 0.112813s 0.075818s
0.251526s 0.067994s 1.716483s 1.637005s 0.113267s 0.075716s
0.250806s 0.067551s 1.721203s 1.626927s 0.113274s 0.075726s
0.251498s 0.067168s 1.716587s 1.651783s 0.113605s 0.076064s

So the ordering would be:

PCRE < CFStringFind < JSRegExp < ICU << RegexKitLite < regex.h

The most complex engine is the fastest! CFStringFind ranks the second probably because my code is not optimized, but this shows the PCRE engine is really efficient. JSRegExp was originally a PCRE with everything irrelevant to Javascript stripped out, but that is slower than PCRE by 2 times. The WebKit devs should think of what's wrong. RegexKitLite's performance is pretty disappointing. Although it is advertised as very efficient, it is still 6~7 times slower than bare-bone ICU. The problem is probably directly using NSString as the RegExp holder, instead of a dedicated RegExp class. And regex.h's result is really a *WTF*.

Edit: I have redone the test to eliminate some inconsistency (case sensitivity) and convert the regex.h's syntax form BRE to ERE. The result is impressive: ERE is faster than BRE by 7 fold, make in in par with ICU. The new results are:

Methods
ICU       PCRE      regex.h   RKLite    JSRegExp  CFStrFind
-----------------------------------------------------------
0.253736s 0.069918s 0.238111s 1.651787s 0.139437s 0.070956s
0.254308s 0.067150s 0.235963s 1.652074s 0.112813s 0.068302s
0.252483s 0.067994s 0.235714s 1.637005s 0.113267s 0.068710s
0.251647s 0.067551s 0.234380s 1.626927s 0.113274s 0.068353s
0.253102s 0.067168s 0.235367s 1.651783s 0.113605s 0.068280s

and the new ordering is:

PCRE ≈ CFStringFind < JSRegExp < regex.h < ICU << RegexKitLite

So PCRE is still the fastest, but with a smaller margin. And RegexKitLite is now win the title of the slowest RegEx implementation.

To conclude this part, we should use PCRE for wildcard testing, if speed and clarity are important.

The source code of the test can be found in http://pastie.org/495990. Basically, 323 URLs collected from 3 different sources are matched against 16 wildcard rules. The file is compiled with:

/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.0 -arch armv6 -std=c99 -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.2.1.sdk -I/Developer/Platforms/iPhoneOS.platform/Developer/usr/include/gcc/darwin/default -I/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.2.1.sdk/usr/lib/gcc/arm-apple-darwin9/4.0.1/include -L/usr/local/lib -Wall -mcpu=arm1176jzf-s -O2 -L. -framework CoreFoundation -framework Foundation -licucore -lpcre main.m RegexKitLite.m ; ldid -S a.out

and run on a jailbroken iPod Touch 1G, firmware version 2.2.1.

Introducing Ad Killer

2009-05-29T07:02:00.005+08:00

This is a short MS extension I've coded up as a direct respond to Greystripe's intrusive advertisement. (How intrusive it is? Download CubesFree and you'll know. C'mon, why not use AdMob?)

Ad Killer is a simple extension that hooks on the NSURLConnection class. It checks the URL against known criteria, and if it is determined as an Ad (or a bad site, whatever), it will convert the URL to a bad site that will always fail the request.

Unlike methods using .hosts, Ad Killer has the potential to support Regular Expressions, and thus AdBlock Plus filters (except element blocking). And unlike AdBlock by CocoaMug, it can work with almost all applications* (not just MobileSafari) and it is free.

Currently Ad Killer is not packaged for release, but you can check out the code at http://code.google.com/p/networkpx/source/browse/trunk/hk.kennytm.adkiller.

*: AdKiller will not work on Cydia, or any setuid apps that does not support MobileSubstrate.

An application that SpringBoard is not tracking just launched with identifier ... and will be killed.

2009-05-28T22:51:00.002+08:00

If you directly run a UIKit app from command line, nothing will be shown, and the syslog will print:

An application that SpringBoard is not tracking just launched with identifier com.yourcompany.Untitled and will be killed.

Why? If you disassemble SpringBoard with thumb-ddis, you can quickly found this string is referred in the function 0x123ba. This function is called only from -[SpringBoard applicationStarted:], decompiled as:

@implementation SpringBoard
-(void)applicationStarted:(GSEventRef)event { _123ba(event); }
@end

So it is pretty easy to get the signature and purpose of this function. 0x123ba decompiled is roughly like:

void LaunchApplication (GSEventRef event) {
 GSEventRecord* pRecord = _GSEventGetGSEventRecord(event);

 struct GSEventAppLaunch {
  pid_t       pid;   // 48
  void*       m_52;   // 52
  const char  displayIdentifier[]; // 56
 }* pAppRecord = pRecord + 1;

 if (getpid() != pAppRecord->pid) {
  // call 11fc8
  NSString* displayIdentifier = NSStringCreateWithUTF8String(pAppRecord->displayIdentifier, pRecord->size - 8);
  if (displayIdentifier != nil) {
   // call 107d8
   SBApplication* app = SBApplicationGetWithDisplayIdentifier(displayIdentifier);
   // 123e8
   if (app != nil) {
    [app sendActivationEvent:event];
    [displayIdentifier release];
    return;
   }
  }
  // 123fa
  NSLog(@"An application that SpringBoard is not tracking just launched with identifier %@ and will be killed.", displayIdentifier);
  kill(pAppRecord->pid, 9);
  [displayIdentifier release];
 }
}

Since we are getting prompt of the display ID there is no reason the condition displayIdentifier != nil will fail, so it must be due to app == nil. We can set a break point at 0x123e8, and indeed, it returns nil. But having the same argument, why one returns a valid object and one returns nil? There must be some global variables it relies on. Let's take a look at 0x107d8 to be sure.

The function 0x107d8 roughly translates to:

SBApplication* SBApplicationGetWithDisplayIdentifier(NSString* identifier) { return _106a0(DisplayStack_e7004, identifier) ?: _106a0(DisplayStack_e7014, identifier) ?: _106a0(DisplayStack_e7008, identifier); }

The DisplayStack_xxx are instances of SBDisplayStack. In this context, SBDisplayStack is like an array of SBApplication, and the function 0x106a0 merely search for an SBApplication having the specified identifier from the stack. So what we are checking is already an aftereffect.

(more about Display stack: http://code.google.com/p/iphone-tweaks/wiki/DevelopmentNotes)

Just for fun: Some undocumented flags of ldid

2009-05-26T22:54:00.004+08:00

ldid was used most commonly to pseudosign an app, so that it would run in a jailbroken device. It is also used to attach an entitlement file to an executable. But besides the well-known -S (pseudosign/add entitlement) and less-well-known -e (dump entitlement) flag, there are a couple of flags went undocumented.

-p <Files>	Print the paths of the input files.
-u <Files>	Print the UUID of the input files.
-t <Files>	Print the timestamp of a dylib.
-T<NewTimestamp> <Files>	Assign the timestamp of a dylib. Pass "-" to <NewTimestamp> to assign the timestamp to a hash.
-s	Calculate the sha1 without changing anything. Incompatible with the -S flag.

Experiment: An alternative way to extract hidden symbols

2009-05-21T22:50:00.002+08:00

A library usually have symbols defined functions and variables to allow external code access its functions, or help debugging. There are external (public) and hidden symbols. The external symbols can be picked up by dyld to broadcast the correct place of the functions. The hidden ones will be ignored, however, for performance reasons, or just to hide some interesting calls.

And the hidden ones are often interesting. For example, WinterBoard needed 7 hidden symbols to work.

Fortunately, the address each symbol points to, hidden or not, is defined alongside the library, so one can read the library file to get all the symbols.

In fact, Apple provided the nlist function for this purpose. But nlist is sloooooow. So slow that saurik has improved the backend of nlist the next version of MobileSubstrate (which delivered more than 10 times speed improvement.)

But how did I step in this? Originally I tried to create a transition filter for QuartzCore. This requires me to access the C++ backend of the QC, which are all hidden symbols. So I use nlist as usual. And I've written a wrapper function using va_arg to make it simpler to use. Then it crashed. Why? There's one caveat — on x86 the QuartzCore is slided. The address returned from nlist — which is just being read statically — is invalid because a run-time offset must be added to it.

This slide can be obtained using the dyld function _dyld_get_image_vmaddr_slide. But it takes an integer! What integer? This is an index of all images loaded by dyld. To check which image it is pointed to, one needs to call _dyld_get_image_name. That's troublesome, and adds extra slowness to the nlist method.

But there's also a _dyld_get_image_header function. This function returns the Mach-O header, which can guide you to do the same thing nlist does, but on the loaded image. Because it is loaded, every read/write will be on the RAM, and it will be much faster than file access. That's good! Except it crashed again — the loaded image is not a direct copy-and-paste. Some part (the __DATA, __OBJC and __IMPORT segments, to be precise) are skipped from the object file. And the same solution for treating QuartzCore cannot be used on UIKit.

So I started to dig the source code of dyld, which I found that these 3 APIs are just C wrapper to a various (i.e. hidden) C++ function. The function, dyld::getIndexedImage, returns a class pointer that has everything I need. Everything. It has the symbol table, the strings table, and the Mach-O header, and that's enough for exploring hidden symbols.

Of course, since dyld::getIndexedImage is hidden, to call it we need to nlist it too... Or not. saurik has already shown that rewriting nlist improves the speed by 10x, why not do it here also? The result, therefore, is this monster:

http://pastie.org/485348.

It has been tested and works on ARM and i386. Experimentally, running the main() function gives the timing in 5 runs:

nlist	lookup_function_pointers
0.517996s	0.026339s
0.498724s	0.013721s
0.498352s	0.013623s
0.501114s	0.013798s
0.498718s	0.015770s

So my method gives up to 30x performance than nlist! I'm happy to include saurik's nlist fix too, but it Bus Errored on strcmp too many times that I've gave up on that. Hopefully later it will be fixed.

Banning SCHelper from syslog

2009-05-09T16:02:00.003+08:00

(Here is the necessary change required, for 2.2.1, if you don't bother to read.)

(Make sure you back up every file you want to change first. We won't take responsibility for your broken devices :p)

syslog is a very useful tool for watching variables and monitoring
system status. But SCHelper make it particular troublesome to follow the messages.

SCHelper stands to System Configuration Helper, and it manages the NSUserDefaults writes. In regular interval SCHelper will synchronize the NSUserDefaults on RAM to the harddisk, and report it in syslog. The problem is, the report takes 7 to 13 lines, and is completely useless to know:


May  9 15:55:16 --- SCHelper[1491]: per-session socket : invalidate fd 9
May  9 15:55:21 --- SCHelper[1491]: processing command "AUTH" w/data
May  9 15:55:21 --- SCHelper[1491]: sending status 0
May  9 15:55:21 --- SCHelper[1491]: processing command "PREFS open" w/data
May  9 15:55:21 --- SCHelper[1491]: sending status 0
May  9 15:55:21 --- SCHelper[1491]: processing command "PREFS lock/wait"
May  9 15:55:21 --- SCHelper[1491]: sending status 0
May  9 15:55:21 --- SCHelper[1491]: processing command "PREFS commit" w/data
May  9 15:55:21 --- SCHelper[1491]: sending status 0 w/reply
May  9 15:55:21 --- SCHelper[1491]: processing command "PREFS unlock"
May  9 15:55:21 --- SCHelper[1491]: sending status 0
May  9 15:55:21 --- SCHelper[1491]: processing command "PREFS close"
May  9 15:55:21 --- SCHelper[1491]: processing command "EXIT"

Usually one watches the syslog using tail -f /var/log/syslog. When SCHelper is in action, suddenly a dozen lines come up and you may miss what you need to know. (Of course you can filter the result using grep, but it's just pretending the problem does not exist.)

To solve the root cause we need to disable SCHelper from yelling. A starting point is to disassemble SCHelper. The file is located at /System/Library/Frameworks/SystemConfiguration.framework/SCHelper. We can search for the keyword processing command in the disassembly, and these line should come up:


 +00104 000023c4 0100A0E3             mov               r0,#0x1
 +00108 000023c8 0710A0E3             mov               r1,#0x7
 +0010c 000023cc 98219FE5             ldr               r2,[pc,#0x198]                    ; -> 0x256c CFSTR("processing command \"%s\"%s")
 +00110 000023d0 960100EB             bl                SCLog (stub)

This is equivalent to the call SCLog(1, 7, CFSTR("processing command \"%s\"%s"), <other arguments>). What, yet another Log function? Hell yes.

This SCLog is an external symbol. We can play a dirty trick to transform all SCLog into ilogb :p. We need to edit the link table for this. The first obvious step is to search for the string _SCLog and replace it to _ilogb (or your favorite pure function). But this will make dyld complain not finding _ilogb in SystemConfiguration (_ilogb is in libSystem.B.dylib). We must change the dylib referenced by this particular symbol as well.

How dyld searches identify a stub symbol? From the Mach-O ABI, we can see it involves a lot of indirection:

Obtain the indirect symbol table from the dynamic symbol table (LC_DYSYMTAB load command)
Every entry of the indirect symbol table points to an index of the symbol table (LC_SYMTAB load command)
An entry of a symbol table is better known as nlist. In our case, the nlist contains reference to the dylib (an LC_LOAD_DYLIB load command) and an offset in the string table.
Finally from the string table we get the actual C string.

What we get is the offset of the C string, _SCLog. We can go up 1 level from there (to the nlist), and modify the dylib it references, then it should work.

For this task, otool and nm is of great help.

Using otool -L SCHelper we can see that the string table (LC_SYMTAB's stroff) is at file offset 15024 (0x3ab0). Additional, the file offset of the string _SCLog is at 0x3cd0. Therefore, the string table offset is 0x3cd0 - 0x3ab0 = 0x220.

Then, in the same load command we can find the symbol table's file offset (symoff) is 12832 (0x3220).

Now, to quickly search for a symbol, we use nm -p SCHelper | nl -v 0. (The -p flags tells nm don't try to sort the symbol table, and the nl part is a Unix utility for line numbering.) We can find _SCLog at 67, meaning we should be able to find the nlist of _SCLog at (0x3220 + 67 * sizeof(struct nlist) = 0x3544).

So let's proceed to 0x3544. The structure of nlist, for our concern is

struct nlist {
  int  symbol_index;
  char lets_not_bother_it_its_always_0x0D;
  char lets_not_bother_it_its_always_zero;
  char lets_not_bother_it_its_always_0x01;
  char the_dylib;
  void* the_vm_address_that_we_dont_care;
};

We can verify the first 4 bytes are indeed 0x220, so this is the correct address. For _SCLog, the_dylib = 2, and the dylib referenced are, in order:

/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
/System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration
/System/Library/Frameworks/Security.framework/Security
/usr/lib/libgcc_s.1.dylib
/usr/lib/libSystem.B.dylib

so we should change the 2 into 5.

I believe there is some link editor that doesn't require us to jump so many hoops. If you know, please leave a comment :).

In summary, the changes are (if you are using 2.2.1):

Open SCHelper with a hex editor.
Head to 0x354B, change the 02 to 05
Search for the string "SCLog", replace it with "ilogb".
Save, and of course, ldid it.

Now respring, and ps -A. Is SCHelper running? Yes, great. Does it emit useless syslog entries anymore? No, it's busy computing logarithms instead :p. Now launch Settings, fiddle with anything. Does it save? Yes, fantastic.

So our task succeeded. Note that if you are using any versions other than 2.2.1, the file offsets may be different.

GriP 0.1-11j released

2009-05-05T15:14:00.003+08:00

This version introduced a lot of features since 0.1-11e (well you see there are 5 private development versions in between, the change is pretty a lot).

Game mode – Suspend GriP in certain applications.
Screen On/Off detection – Hold important messages when the screen is off, and show them all when the screen is on again.
Multi-column support – Basically, you can fill a whole screen of GriP messages where you could only fill half before :p
Theme customization – Currently you can only change the width of the Default theme, but the framework is completely general.

For developers, GriP now supports using Emoji as icon. For theme makers, the max width of the GriP message window can be queried using `+[GPMessageWindow maxWidth]`. The background colors of the window can be modified with `+[GPMessageWindow setBackgroundColor:]`, if you don't like to do that yourself.

You can download at http://code.google.com/p/networkpx/downloads/list as usual.

Introducing CFLog

2009-05-02T04:00:00.006+08:00

In Foundation, there is a very handy printf-like function for debugging, called NSLog. You can call it like this:

NSLog(@"%@'s temperature is %f K", [NSDate date], 310.f);

In CoreFoundation, however, there is just a single-argument CFShow. This forces the simple NSLog line split into 3:

CFStringRef debugString = CFStringCreateWithFormat(kCFAllocatorDefault, NULL, CFSTR("%@'s temperature is %f K"), [NSDate date], 310.f);
CFShow(debugString);
CFRelease(debugString);

This is very annoying when you have lots of code like these. Of course we can always define a macro or function to take out all the repeating parts, e.g. in MobileSubstrate's source code:

#define CFLog(args...) \
    do { \
        CFStringRef string(CFStringCreateWithFormat(kCFAllocatorDefault, NULL, args)); \
        CFShow(string); \
        CFRelease(string); \
    } while(0)

But there is actually a more powerful CFLog built into CoreFoundation. It is, unsurprisingly, called CFLog:

/*
        APPLE SPI:  NOT TO BE USED OUTSIDE APPLE!
*/

enum { // Legal level values for CFLog()
    kCFLogLevelEmergency = 0,
    kCFLogLevelAlert = 1,
    kCFLogLevelCritical = 2,
    kCFLogLevelError = 3,
    kCFLogLevelWarning = 4,
    kCFLogLevelNotice = 5,
    kCFLogLevelInfo = 6,
    kCFLogLevelDebug = 7,
};

CF_EXPORT void CFLog(int32_t level, CFStringRef format, ...);

The header code can be found in http://www.opensource.apple.com/darwinsource/10.5.6/CF-476.17/CFLogUtilities.h (requires login). This is available on the iPhoneOS too as an undocumented function starting from 2.2. So you can now do it like this instead:

CFLog(kCFLogLevelDebug, CFSTR("%@'s temperature is %f K"), [NSDate date], 310.f);

Simple and beautiful.

GriP (Growl for iPhone) is in beta now.

2009-04-24T11:28:00.004+08:00

Let me cut it short: you can download GriP 0.1-11b (beta) from http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.grip-0.1-11b.deb now. The documentation is also available in http://code.google.com/p/networkpx/wiki/GriP, which contains most thing you are interested. It should run flawlessly on a jailbroken 3.x device too, but I haven't tested (I don't have a 3.x device right now).

self notes...

2009-04-05T02:08:00.003+08:00

[UIFont smallSystemFontSize] == 12
[UIFont systemFontSize] == 14
[UIFont labelFontSize] == 17
[UIFont buttonFontSize] == 18

Default font size of Rounded Rect button == 15
Default font size of UITextField == 12

Releases: Thumb Dumb Disassembler & Calculator.searchBundle

2009-03-31T00:52:00.003+08:00

Last few weeks I've diverted myself from iKeyEx to the 3.0 VFDecrypt key. The result? Of course it's failure, otherwise you'll see the key on theiphonewiki right now.

But a nice by-product is the "Thumb Dumb Disassembler". For iPhone reverse engineering, if the code is compiled to ARM then ravel-arm can be used to give very useful information. However, if it is compiled to Thumb then ravel can't handle the code correctly. It will treat the code as ARM and output garbage. Sadly, unlike otool, there is no force-Thumb mode in ravel so we can't do much.

When nobody can save you, you have to save yourself. Therefore I've written a disassembler specially for Thumb mode. This disassembler can extract useful data and perform numerical arithmetic linearly (ignoring all branches). I term this Dumb Disassembler as it doesn't perform branch analysis nor symbolic arithmetic. A Smart Disassembler will do both and the result in decompiler-quality output.

The Thumb Dumb Disassembler can be downloaded in thumb-ddis.zip. Unlike other networkpx projects, Thumb Dumb Disassembler is released in GPLv3.

(Sidetrack: What about the VFDecrypt key? In pre-3.0 asr there is a specific __DATA,__restore section to store the key. In post-3.0 asr this is computed in run time using the SHA-1 and SHA-256 keys of the CPU identifier (s5l8900x) and the content of the whole ramdisk. And then I got a 64-char incorrect password from it. There is another way to extract the key: run the asr, put a break point at 0x00011836, and retrieve the CFString at r0. On my device both the 2.2.1 and 3.0 asr Bus-errored me when I try to actually run them.)

Now, let's talk about something not so technical. One of the new customer features in iPhoneOS 3.0 is the Spotlight. But one essential component missing from the Mac OS X Spotlight is the calculator.

When the SDK was just released, some developers noticed the directories /System/Library/SearchBundles/*.searchBundle and it does mean Spotlight is extensible right? Indeed it is.

After disassembling the 2 searchBundles in it, I have a rough idea of how these bundles work. So with minimum effort, I've created this Calculator.searchBundle:

Because 3.0 is not jailbroken yet, only the Simulator version exists. If you have the SDK, you can now download Calculator.searchBundle.zip to test the bundle. The source code is also provided (under BSD license) so you can code up your own.

Note that the 3.0 SDK is not finalized so the code shown here may not work in June.

3.0 Keyboard Changes

2009-03-21T14:19:00.003+08:00

The keyboard will now transit (by fading & resizing) between landscape and portrait mode, i.e. the keyboard no longer hides when the orientation is changed (at least in Notes)

Emoji got landscape mode

Accent view and popup view got less shadows -- I don't like this.

When disabling an internal keyboard that was active, the OS no longer reset the whole keyboard list. This means fewer steps to replace internal keyboards by iKeyEx ones. ^_^

I've uploaded a gallery of the extra keyboards and interesting features in 3.0

Some points worth noticing:

Even though Arabic and Hebrew are RTL languages Apple doesn't bother to reverse the direction of the delete key. Probably they think it is not very confusing.

In Zhuyin, after you entered a consonant in the normal plane, it will automatically switch to the Shift plane for the vowels.

In Greek, the final sigma is automatically detected, so you don't see a final sigma on the keyboard.

I don't know if they're still using a standard keyboard (i.e. simple subclass of UIKeyboardLayoutRoman) for Thai and Zhuyin as the position of keys in normal plane and Shift plane are distinct. This cannot be done in 2.x at all.

Also I don't know if the accents view becoming multi-row is internal or not.

Now excuse me trying to JB the 3.0 and extract the framework binaries...

New Keyboards in 3.0:

2009-03-18T23:56:00.002+08:00

Arabic,
Hebrew,
Malay,
Thai,
Chinese (Traditional) (Zhuyin 注音)

Still no Chajei 倉頡 :(

Nice job on Copy and Paste Apple,

2009-03-18T03:17:00.001+08:00

But I hate shake to undo :p Nor does all the UIAlertViews for push notification.

iKeyEx & 5-Row QWERTY 0.1-9b (Beta) Released

2009-03-17T17:33:00.008+08:00

First of all, about the versioning schemes -- From now on all beta versions (those I put in the project download page and my private beta Repo) will have an odd revision, and the public versions will have an even revision (those you see in BigBoss). Therefore, the following beta versions will be called 0.1-9c, -9d, etc., and the release version will be 0.1-10.

iKeyEx

So back to iKeyEx. What's new in 0.1-9b has actually been highlighted in the last post. I'll go to the detail and implementation of each feature now.

More than 10 variants

Prior to 0.1-9b the number of variants supported must be less than 10 because if there's more the buttons will leak off the screen and become useless. In 0.1-9b the system will automatically reduce the size of each button to try to allow more keys to be shown.

The upper limit is now pushed to 12. Larger than 12 the variants list will start to act weirdly.

Variants Labeling

Following layout.plist, the variants can also be "labeled", i.e. the text shown on the button can be different from what will actually be typed. This is done by declaring an array instead of a string, e.g.

M = (M, ("Dear all,\n\n\n\nBest Regards,\nMe", Mail))

will create a Mail button on the list of variants, and when selected, will generate the "Dear all, ..." text.

Splitting of Landscape and Portrait Mode

The type of UIKeyboardLayoutCLass key has been generalized. As a result, not only code can have separate layouts in landscape and portrait mode, the other two methods (layout.plist and referred) can be split too.

The syntax is simple:

UIKeyboardLayoutClass = {
  Portrait = "abc.plist";
  Landscape = "xyz.plist";
};

Keyboard Mode Jumping

An extra feature in iKeyEx -- you can now hold down the International button (the Globe) for more than 1 second, then release it to get a list of keyboards enabled.

Click on an item to go directly to that keyboard.

Note: This feature is not available in the Emoji keyboard. 3rd-party IMEs not based on iKeyEx cannot be identified.

Note: In 0.1-9c you don't even need to release the button -- just keep pressing it and the keyboard list will show automatically. But in 0.1-9b you still need to lift your finger.

Email Diagnosis Info

The last, and the most important feature in this upgrade is the Email Diagnosis Info button in Settings -> iKeyEx -> Troubleshooting. I've got lots of crash reports but the SSH process is pretty troublesome for most users. Therefore I've created this button to simplify the process. The files attached are:

Recent crashes related to iKeyEx (i.e. /User/Library/Logs/CrashReporter/*.plist which have the substring "iKeyEx.dylib␣␣" in the file)

The list of installed Cydia packages (i.e. dpkg -l)

syslog (/var/log/syslog)

.GlobalPreferences.plist for the list of enabled keyboards (/User/Library/Preferences/.GlobalPreferences.plist)

com.apple.Preferences.plist for the active keyboard (/User/Library/Preferences/com.apple.Preferences.plist)

The content of /User/Library/Keyboard/ and /Library/iKeyEx/Keyboards/

This also serves as a demonstration on how to send email without leaving the application, and how to attach files and data to the message.

etc.

There are some minor tweaks in this version, including:

The /Library/iKeyEx/ folder is now a symlink to /var/stash/iKeyEx.XXXXXX/ to free up precious space in the / partition.

An "InputManagers" folder is created besides "Keyboards" folder. Eventually in 0.2 the input managers will be split from keyboard layouts and allowed to mashed up, but now this is just an empty placeholder.

All keyboards with name starting with __ are reserved for internal use now.

5-Row QWERTY Keyboard

5-Row QWERTY is also updated, but it is basically a bug-fix update. The ChangeLog has already summarized the changes.

One week of inactivity?

2009-03-14T03:34:00.003+08:00

The networkpx project was inactive for the whole week. Why? If you see the last post you should know I was trying to implement a screen video capturer. Eventually it's a miserable failure. The problem is lack of RAM -- the capturing program uses too many resources and caused a kernel panic. So I have eventually got rid of the idea. Instead, a viable solution would be capturing through the USB cable with the MobileDevice (private) framework. I don't know MobileDevice, so let's put it aside.

So what now? I'm upgrading iKeyEx to the next minor version 0.1-10. Features included or expected to include are:

Squeezing the variants when there's more than 10 of them, so Vietnamese accents can be shown properly (implemented)
Allow disassociating portrait and landscape mode in layout.plist
"Email diagnosis info", which automatically sends all the crash log to me by mail. I create this because many issue reporters failed to give me the info I really needed :( so let's automate it.
Long press the globe to select a keyboard to jump to.
Labels for variants.

This will not be the last version of the 0.1 line because there are still numerous crash reports floating around. The "Email diagnosis info" will be the key feature that allows more accurate crash data collection. Then hopefully 0.1-11 will be the last of 0.1-*.

(if you're a (non-SDK) developer and wants to implement similar feature, you can look into src/MailCrashLog.m and src/UIKit3/UIMailComposeView.m of the SVN trunk now.)

iPhone Screen Video Capturing

2009-03-10T02:15:00.003+08:00

(Yes, I know the colors are all wrong, and the time sequence is strange, but these are very easy to fix.)

(The video is blurred solely because of conversion.)

To move folders in / to /var/stash/

2009-03-07T00:49:00.004+08:00

Add this line to preinst:


if [[ $1 == install ]]; then
  /usr/libexec/cydia/move.sh <FULLPATH-YOU-WANT-TO-MOVE-FROM>
  ...
fi;

But it seems no way to clean it up.

Arbitrary HTML in Google Code wiki

2009-03-03T21:49:00.002+08:00

Google Code supports a limited subset of HTML in Wiki. In particular, <input/> cannot be used. Because of this, the early version of CopyingTextFromSafari uses the traditional "go to a link and bookmark it" mechanism to add the bookmarklet.

Then today I realized that I could use HTML in Google Gadgets. So I can just

Put all HTML in an XML file and upload the file to SVN.
Insert a <wiki:gadget/> with URL to the wiki page.

I've immediately changed CopyingTextFromSafari, and also added a Donate button to the front page after figuring this out.

One slight note is that the HTML you use will live inside an iframe.

0.1-8

2009-03-03T16:20:00.004+08:00

I've sent iKeyEx and 5-Row QWERTY (which isn't 5-Row nor QWERTY anymore :p) 0.1-8 to BigBoss. Yes this time is real.

iKeyEx 0.1-8 fixes a critical crash on pre-2.2 firmwares. Users of pre-2.2 users should upgrade to iKeyEx 0.1-8 (or just upgrade to firmware 2.2.1!). Hopefully all crashes will be solved soon so that I can end the 0.1-* chronology.

5-Row QWERTY 0.1-8 adds a lot of customizations. Pretty much of these have been covered on iClarified.

Issue 67

2009-03-01T20:34:00.005+08:00

iKeyEx 0.1-6 crashes some machines. This is a known problem, a very critical problem, but I did not know how to reproduce it. Fortunately, a proper bug report finally appear as issue 67 and I got to know what has happened. Turns out it's a firmware issue. In ≥2.2 there is a message named

-[UIKeyboardSublayout setIsShiftKeyPlaneChooser:]

which does not present on ≤2.1. In Objective-C, any unrecognized messages will cause an exception to be raised. Since I did not setup any exception handlers, the result is obvious -- crash.

There is another piece of message that is new in 2.2:

+[UIHardware _playSystemSound:]

These 2 should make hClipboard and iKeyEx-derived keyboards unusable in ≤2.1.

I should have caught these when I claimed issue 7 is solved. But why not? This is pretty much an issue of how Objective-C work.

Issue 7 was filed because I found that the linker for 2.0 and 2.1 emitted errors. All external C and C++ functions must be processed by the linker, and if something is missing (even if the header file said it was there), we can immediately know. But Objective-C is different. Objective-C dispatches message implementation dynamically. That's why Category and method swizzling works. But this flexibility means no compile-time check can be done. And the runtime check causes crashes.

What I've done immediately after recognizing the root cause of issue 67 is to build the API diff table. This allows developers to easily see what's new and what's missing during firmware transition, and determine whether a respondsToSelector: call is worth it.

Meanwhile, iKeyEx v0.1-8 fixing the 2 calls has been uploaded to Google code as well.

2009-03-01T18:42:00.006+08:00

The problem is fixed. Install hClipboard as you wish :)

The "latest" version of hClipboard seems to be mispackaged. While it is shown as 0.1-5, the content is actually 0.0-1, the initial buggy version.

I don't know what happened. I haven't sent any new packages to any repositories after Feb 19th.

Please do not "upgrade" to this version, because you are in fact downgrading.

If you have unfortunately "upgraded", you can remove that package, then install 0.1-5 from http://code.google.com/p/networkpx/downloads/detail?name=hk.kennytm.hClipboard-0.1-5.deb.

0x51EB851F wha?

2009-02-27T18:54:00.005+08:00

I am decompiling KBWordSearch for .cin support, but I encountered a strange piece of ASM:

+00174 30b9536c 34329FE5 loc_000174: ldr r3,[pc,#0x234]
+00178 30b95370 C7208BE2 add r2,fp,#0xc7
+0017c 30b95374 9302C1E0 smull r0,r1,r3,r2
+00180 30b95378 C23FA0E1 mov r3,r2,asr #31
+00184 30b9537c 41B363E0 rsb fp,r3,r1,asr #6
+00188 30b95380 30109DE5
+0018c 30b95384 8B32A0E1 mov r3,fp,lsl #5
+00190 30b95388 8B3183E0 add r3,r3,fp,lsl #3
+00194 30b9538c 033183E0 add r3,r3,r3,lsl #2
+00198 30b95390 02B063E0 rsb fp,r3,r2

which translates to:

// in: index
// out: next_index
capacity = 200;
r2 = index + capacity - 1;
r1 = r2 * 0x51EB851F; // take upper dword only.
fp = (r1 >> 6) - (r2 >> 31);
next_index = r2 - fp*160;

(0x51EB851F is the lower dword of 3.14 in IEEE double, and also 2³⁷/100+1) Can someone explain what this code is doing?

Edit: Turns out to be a division. Ref: http://d.hatena.ne.jp/h0shu/20080302/p2

r2 = index + capacity - 1;
next_index = r2 - (r2/capacity)*160;

Why are they doing things like this is beyond me.

iKeyEx & 5-Row QWERTY 0.1-7 Released

2009-02-27T00:59:00.007+08:00

I was not updating last few days because I wanna clear all the homework first. Anyway, there's no major update about ⌘, but there is a huge addition in 5-Row QWERTY.

The front end to modify Info.plist & layout.plist of 5-Row QWERTY has been implemented in v0.1-7, so that you can:

Change the auto-correction language, or even use the built-in IMEs:
(5-Row QWERTY × Traditional Chinese Pinyin IME)
Change the default layout to QWERTZ, QZERTY, AZERTY and many others
Totally customize the special characters, e.g. change the ^ to ?, $ to €, etc.

There are a few bug fixes and improvements as well, which are recorded in the ChangeLog as usual.

iKeyEx 0.1-7 is a minor update which addresses a few bugs. However, the critical one still has not been fixed because no one has ever given me a crash log or a useful crash log.

Although the layout engine of 5-Row QWERTY only depends on 0.1-6, the preferences binary depends on a single 0.1-7 function "iKeyEx_KBMan" for clearing the cache quickly. Therefore you still need to upgrade iKeyEx to make it work.

⌘ Development Progress Day 3

2009-02-22T02:54:00.002+08:00

Nothing technical, so let's see some screenshots:

Spelling error slipped into production

2009-02-21T13:57:00.000+08:00

Open UIKit (2.2), and search for UIResrouceBundleForNIBBeingDecodedWithCoder. Lol.

⌘ Development Progress Day 2

2009-02-20T19:00:00.003+08:00

Now that UILabel is all set, I continue to hook other texts that are long enough to be "commanded", e.g. the UITableHeaderFooterView. There are also some reluctant UILabel that refuses to be hooked. For these I hooked their superviews (UINavigationBar & UIPickerTable) for extracting the text info.

And here comes the real challenge: hooking the UIWebDocumentView. Yes I knew the Clippy 0.95-5 experiment was failed. But this does not prevent anyone else from finding another way to achieve the same goal right?

My first naïve approach is to get the DOM node at the touch point, like this:

[[self webView] elementAtPoint:(touchPoint)];

and like Clippy 0.95-5, it crashes immediately at the 2nd touch. If I trace the class of the returned object*, I can see the class changes from e.g. DOMHTMLAnchorElement to NSCFType. In fact I encountered this before once when trying to implement setSelection() to <textarea/> and <input/> in ℏClipboard using the DOMRange object. It fails on the 2nd call. I call this the "observer effect". Right after you "observed" any DOM objects, the object will "collapse" and you can't reference it reliably anymore.

A workaround is to -retain it right after the DOM object is accessed. But this introduces memory leak. This is really a last-resort option.

And actually it turns out to be work done too complicated. There is a -[UIWebDocumentView approximateNodeAtViewportLocation:] method in the Interaction category which can easily be overlooked. It returns a DOMNode without any observer effect AFAIK. Except it crashes on 2.2 complaining ASSERTION FAILED: !WebThreadIsEnabled() || WebThreadIsLocked(). Well that's easy enough to solve. Just lock the WebThread() everytime we need to access this method.

So I used:

WebThreadLock();
DOMNode* activeNode = [self approximateNodeAtViewportLocation:&(touchPoint)];
WebThreadUnlock();

and done! The DOMNode is got and we can do anything we want. Too good to be true. Turns out the DOMNode will only be returned on any objects that have actions, i.e., <a>, <input/>, <textarea/>, <img/>, any thing that has onXXX events, but not static texts. If touchPoint is on a static text, nil will be returned.

But why this method prevents the observer effect? After lots of experiments, finally it turns out that it's the WebThreadLock() and WebThreadUnlock() doing the tricks. In fact, if I do

WebThreadLock();
[[self webView] elementAtPoint:(touchPoint)];
WebThreadUnlock();

then everything works perfectly! At least in the simulator. (Hey Ryan, if you're reading this, as now I've solved this problem, can you add Safari copying back in Clippy 0.95-6? :D)

So, at the end of Day 2, I can hook to any DOMNodes, besides UILabel, UITableHeaderFooterView, UINavigationBar and UIPickerTable.

Note: -[WebView elementAtPoint:] returns a subclass of NSDictionary. By the returned object I mean the value corresponding to the WebElementDOMNode key.

⌘ Development Progress Day 1

2009-02-19T23:19:00.004+08:00

OK, now iKeyEx 0.1-6 has been released, let's turn into the other project left over: ⌘.

⌘ was proposed on Jan 26th as a solution to issue 10. You can read the design concept here. As I develop the app, I'll keep updating the progress on this blog.

The first thing I've done is to intercept touches coming to a UILabel using -[UILabel touchesBegan:withEvent:]. It turns out not so easy. By default UILabel ignores all touches. But setting userInteractionEnabled = YES does not solve the problem at all (at least in 2.0 firmware). As usual, when something does not work, I look into the UIKit's assembly code. And it turns out that there is a -[UILabel ignoresMouseEvents] method. So I quickly hooked it to always return NO. Then the test application crashed, complaining -[UIScrollView old_touchesBegan:withEvent:] not found. But hey, I did not do anything to the UIScrollView! What's wrong? Turns out the implementation of -[UIResponder touchesBegan:withEvent:] looks like this:

-(void)touchesBegan:(NSSet*)touches withEvent:(UIEvent*)event {
forwardMethod2(self, _cmd, touches, event);
}

where forwardMethod2() is equivalently the statement [[self nextResponder] performSelector:_cmd withObject:touches withObject:event]. This pattern is used for all touchesXXX:withEvent: messages. Now since I have renamed the old touchesBegan:withEvent: for UILabel only, and my code forwards the message to the next responder (the UIScrollView), and it does not recognize old_touchesBegan:withEvent: -- blam, crash.

As a result, I have to store the old IMP before MobileSubstrate-ing it, and invoke the old message like this:

-(void)touchesBegan:(NSSet*)touches withEvent:(UIEvent*)event {
NSLog(@">>> %@", self.text);
old_touchesBegan_withEvent(self, _cmd, touches, event);
}

With this, I can hook all UILabels in the simulator.

Upgrades

2009-02-19T08:07:00.002+08:00

I've sent the latest versions (r167) of iKeyEx, ℏClipboard, 5 Row QWERTY, L33tTyper and MathTyper to the BigBoss. Hopefully they'll be available tonight.

5 Row QWERTY 0.1-6 released

2009-02-18T11:33:00.004+08:00

5 Row QWERTY 0.1-6 is released, and iKeyEx is also upgraded to 0.1-6 to support the features of it.

I've put up a few cached images of the new layout at http://code.google.com/p/networkpx/wiki/Using_5RowQWERTY.

BTW, if there's no more blocking issues, the new versions of iKeyEx, ℏClipboard, 5 Row QWERTY, MathTyper and L33tTyper should hit BigBoss tomorrow morning (GMT+8).