Few cents about my commits

RoboVM 2.3.10-SNAPSHOT, experimental build

|

@Tomski made first 2.3.10-SNAPSHOT build today. It artifacs deployed to sonatatype. Idea snapshot plugin is available for download.

PLEASE NOTE:

  • IMPORTANT: Idea plugin is now a .zip file and Safari will unpack it (by default) which will make it not functional. It has to be downloaded as .zip file and installed using .zip file, not with .jar file inside zip/unpacked folder.
  • At moment of writing latest release is 2.3.8 and latest snapshot is 2.3.9-SNAPSHOT.
  • 2.3.10-SNAPSHOT is experimental as contains many introduced changes in compier/api/bindings which were not deeply tested. While 2.3.9-SNAPSHOT version is being kept for possible hotfixes.
  • 2.3.10-SNAPSHOT is build from jdk12 branch and it names might be confusing. It doesn’t deliver JDK12 functionality to RoboVM but was opened to allow RoboVM compiler be running on JDK9+. As result it hosts 2.3.10 version.

What’s changed

Source tree: migrating to jdk12

  • maven project structure reformated;
  • removed sonatype root parent pom;
  • improved dependency management (moved to root pom);
  • plugin versions/dependencies updated to recent where possible;
  • unit tests fixed to run on jdk9+/openjdk.

Bindings

Compiler changes (fixes and new tricks)

  • Generic class arguments and @Block parameters: PostFix #1, pr419
  • Support for non-static @Bridge method in enums classes: PostFix #2, pr420
  • Support for @Block member in structs: PostFix #3, pr421
  • [fixed]Compilation failed on @Bridge annotate covariant return synthetic method: PostFix #4, pr422
  • Support for Struct.offsetOf in structs: PostFix #5, pr431
  • workaround for missing objc classes(ObjCClassNotFoundException), PostFix #8, pr442
  • Hooray! experimental and formal bitcode support, PostFix #9, pr443
  • [fix] error code -34018 when using Security API on simulator: pr447, post

Idea plugin

  • fixed annoying Android gradle faced #242, post;
  • plugin is now built with gradle script, see migrating to jdk12;
  • code was cleaned up, upgraded to Java8 code level, and deprecated API usage were removed where possible to support Idea 2019.3, “no Xcode dialog” is not blocking anymore: pr434;
  • disabled generate separate IDEA module per source set: pr449, post;

Happy coding! Please report any issue to tracker.

Idea: disabling `generate separate IDEA module per source set`

|

Idea 2019.2 deprecated generate separate IDEA module per source set option as mentioned in their blog post, reworked Gradle settings dialog. For RoboVM this option is important as it is required to be disabled. Having it enabled it will produce three modules once gradle file is imported:

module (content root: ./ )
    module-main (content root: ./src/main)
    module-test (content root: ./src/test)

Root case

Its a problem for Idea plugin to find out the module eligible to run. As one of criteria is presence of robovm.xml in content root of module. Example above shows that module with code is module-main but its content root points to source folder not module root folder itself (and there is no robovm.xml). Group module module contains valid content root with robovm-xml but no source root. As result run/debug dialog shows no valid module to run/debug.

Workaround

Simulator: enabling Security framework(fix for error -34018)

|

@Tom-ski reported that Security api like SecItemAdd fails in RoboVM applications while running on simulator with error -34018:

The operation couldn’t be completed. (OSStatus error -34018 - Client has neither application-identifier nor keychain-access-groups entitlements)

Same time there is no issue for application when running on device. There are multiple reports about similar issues on web, and one on Stackoverflow contain reference to statement from Xcode8.1 release notes:

Keychain APIs may fail to work in the Simulator if your entitlements file doesn’t contain a value for the application-identifier entitlement.

Quick test reproduced this issue with Java code like this:

SecAttributes attributes = new SecAttributes();
attributes.set(SecQuery.Keys.Class(), SecClass.Values.GenericPassword());
attributes.setAccount("My key");
attributes.set(SecValue.Keys.Data(), new CFString("My secretive bee 🐝"));
attributes.setAccessible(SecAttrAccessible.WhenUnlocked);
try {
    CFType res = SecItem.add(attributes);
    System.out.println(res);
} catch (OSStatusException e) {
    e.printStackTrace();
}

But there is an error possible in bindings, to check this native library with simple class was generated and called from RoboVM project:

@implementation SimpleTest
-(void) test {
    NSMutableDictionary *queryAdd = [NSMutableDictionary dictionary];
    [queryAdd setValue:kSecClassGenericPassword forKey:kSecClass];
    [queryAdd setValue:itemKey forKey:kSecAttrAccount];
    [queryAdd setValue:[itemValue dataUsingEncoding:NSASCIIStringEncoding] forKey:kSecValueData];
    [queryAdd setValue:kSecAttrAccessibleWhenUnlocked forKey:kSecAttrAccessible];
    int resultCode = SecItemAdd(queryAdd, nil);
    NSLog(@"SecItemAdd %d", resultCode);
}
@end

And result was same error -34018.

the fix

Same time this ObjC code successfully runs on simulator from Xcode. That means there is something in build/signing process missing in case of RoboVM. Looking throw Report navigator shows following:

  • Xcode signs simulator binary with ADHOC identity. But it is not enough;
  • Xcode adds entitlements during signing. But signing with simple get-task-allow (as mentioned in web) crashes App on start with Invalid entitlements exception.
  • It turns out that Xcode uses com.apple.security.get-task-allow instead. With it app doesn’t crash but Security API still doesn’t work;
  • Inspecting binary built by Xcode for application-identifier discloses another entitlements (besides one in signature) stored in __TEXT __entitlements and it contains following:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>application-identifier</key>
	<string>6THE3YW9HQ.bundle.id</string>
	<key>keychain-access-groups</key>
	<array>
		<string>6THE3YW9HQ.bundle.id</string>
	</array>
</dict>
</plist>

Having such section embedded into RoboVM application solves the issue. Success !

Bottom line

As result:

  • signing of binary built for simulator is not required, but was added to be compatible with what Xcode is doing;
  • RoboVM compiler now embeds __entitlement section similar to Xcode (but this is not documented anywhere).

Code was delivered as PR447

iOS13 PostFix #10: glkit -- missing functions (static inline now)

|

Report in gitter shown another problem in bindings: java.lang.UnsatisfiedLinkError: Optional @Bridge method GLKMatrix4.create(FloatBuffer;)GLKMatrix4; not bound

Quick investigation shown that this method is not available runtime anymore as tons of functions converted into static inline. Today this functions looks as bellow:

GLK_INLINE GLKMatrix4 GLKMatrix4MakeWithArray(float values[16])
{
    GLKMatrix4 m = { values[0], values[1], values[2], values[3],
                     values[4], values[5], values[6], values[7],
                     values[8], values[9], values[10], values[11],
                     values[12], values[13], values[14], values[15] };
    return m;
}

As result RoboVM is not able to resolve it runtime and use it. As long as it is not available runtime there are two options to use it:

  • get all API and .c file, build it into object file and distribute as part of bindings;
  • implement missing API in java (porting it).

While first option is preferable due performance concerns second approach will be implemented as quickest solutions. All changes were added to ios13.2 bindings branch as commit.

Other postfixes:

iOS13 PostFix #9: experimental and formal bitcode support

|

Bitcode was considered as big question once Apple introduced support for it. Mostly it was seen as danger for RoboVM project as Apple might use it as a blocking requirement and not allow application to be submitted without bitcode. While today lot of people think bitcode is not deliver significant benefits (if apple re-compiles apps on own servers) having it might be good idea due:

  • developers need it due fear;
  • it is required when producing Framework target with RoboVM and including framework in bitcode-enabled project;
  • it was a part of commercial RoboVm 1.14 compier.

How support works

There is nice post Embedded Bitcode by Jonas Delieghere that covers basics. In case of RoboVM getting bitcode is not big deal as its an optimized form of LLVM IR that RoboVM compiler produces while building native code. The question is how to get it emited to object file. From post there are two option to mark object file as having bitcode:

  • Full support: a section __LLVM, __bitcode for storing the (optimized) bitcode. This is plain, binary bitcode, not wrapped in an archive as there’s only one file for each object;
  • Formal support: an empty section __LLVM, __asm to differentiate objects without bitcode from those built from assembly.

This post fix implemented formal support due following moment:

  • to add bitcode as section to obj file would require implementing functionality similar to clang’s -fembed-bitcode as LLVM either produces bitcode, object or assemble file;
  • RoboVM compiler does produce object file from assember file (it produces it from LLVM IR to hack method size fields in info object);
  • genereting bitcode and allowing Apple to re-compile app will break line number and method information. As it being received from DWARF debug one, once recompiled by apple – offsets might not match anymore. And it would require to disable line number information in stack traces.

What was done

  • native libraries were re-compiled with -fembed-bitcode flags;
  • each object file generated from Java class now gets __LLVM, __asm section;
  • linker is being instructed to use -fembed-bitcode;
  • as runtime libraries are huge now due included bitcode – it being stripped of if not enabled to minimize debug footprint;

How to use

Bitcode support can be enabled:

  • in robovm.xml with <enableBitcode>true</enableBitcode>;
  • in robovm setion of build.gradle with enableBitcode = true;
  • in enableBitcode parameter to maven plugin;

Code is delivered as PR443

Other postfixes:

iOS13 PostFix #8: workaround for missing objc classes(ObjCClassNotFoundException)

|

Apple altogether with introducing new APIs also keeps refactoring existing one. In this case it results in part of API being extracted from class and moved to newly introduced class. And this class is being used as new super. Result class will have schema when super class introduced later than child. This makes problem when running code on iOS older than super was introduced. Bright example:

// since ios 4.1
@interface GKPlayer : GKBasePlayer

// but

// since ios 10.0
@interface GKBasePlayer : NSObject

Root case

iOS13 PostFix #7: bindings for ios13.2

|

This post continues the series of fixes discovered during compilation of CocoaTouch library and improvements to compiler.

PostFix #7: iOS 13.2 bindings

While things being fixes iOS 13.2 is already in wild. This postfix delivers:

  • changes to support new api;
  • fixes in javadoc;
  • fixes to structures that have to be annotated as packed.

Code is delivered as PR441

Other postfixes:

iOS13 PostFix #6: Fixes to Network framework bindings

|

This post continues the series of fixes discovered during compilation of CocoaTouch library and improvements to compiler.

PostFix #6: Fixes to Network framework bindings

As metntioned in gitter channel a try to use NWPathMonitor fails with ObjCClassNotFoundException: OS_nw_path_monitor. The binding of Network framework differs from common approach in way it is completely functional interface. Most of API might be simply grouped into two groups:

  • creators: CLASS INST = function_Create()
  • usage: function_Usage(CLASS INST, other params) RoboVM bro compiler allow to use such functions as class memebers, e.g.:
    function_Usage(CLASS INST, other params)
    to
    class CLASS {
      @Bridge
      function_Usage( other params)
    }
    

Same time bro-gen extracts types defined in framework as protocols (like NWPathMonitor is defined as OS_nw_path_monitor) and closes binding for it is a NativeProtocolProxy. Which shall work in general case but not this.

RoboVM 2.3.8 released, what's new

|

What’s new

  • [fixed] #408 Cannot compile AUMIDIEvent;
  • [fixed] #414 Deployment to ios13 device, updated libimobiledevice lib’s bindings;
  • [fixed] swiched to simctl to support Xcode11 simulator PR#410;
  • [improvement] kotlin improvements: debugger and compilation PR#349
  • [fixed] #387 CGBitmapContext.create fails on real device for big sizes;
  • [added] support for packed structures PR#378;
  • [fixed] Broken enum marshallers cleanup PR#377;
  • [fixed] Debugger crashed when native thread exited while paused on breakpoint PR#384;

What’s in progress

Meanwhile work in progress happening in support for JDK12 tools on host branch, it contains following changes:

  • all builds scripts reworked to allow building RoboVM using JDK9+ (works with JDK13) PR#400;
  • ios13 bindings PR#406;
  • Idea plugin workaround for long-going android gradle faset #242, PR#401;
  • On-going series of Cocoa13-postfixes

Happy coding!
Please report any issue to tracker.

iOS13 PostFix #5: implementation of missing Struct.offsetOf

|

This post continues the series of fixes discovered during compilation of CocoaTouch library and improvements to compiler.

PostFix #5: Struct.offsetOf always returns zero

Other postfixes:

Struct.offsetOf is required to proper implement initialization of variable size structs with flexible array member such as:

struct vectord {
    size_t len;
    double arr[]; // the flexible array member must be last
};

Root case and fix

Struct has offsetOf definition and it returns always zero similar to sizeOf method. `` its implemenation is being synthesized by RoboVM compiler and invocation of sizeOf being fixed during trampoline phase from Struct.sizeOf to DestStruct.sizeOf (DestStruct – an example struct implementation). Root case – compiler doesn’t not synthesize offsetOf.