Few cents about my commits

tutorial: RoboVM and dependent watchOS app

|

tutorial: marshaling NSArray of protocols

|

Backtrack: issue, gitter

It becomes problematic to marshal methods that return NSArray<id<PROTOCOL>> like bellow:

- (void)readerSession:(NFCReaderSession *)session 
        didDetectTags:(NSArray<__kindof id<NFCTag>> *)tags;

Issue 1: NSArray

Direct binding will not compile:

fixed: idea debug session hang + launcher classes rework

|

Issue

There was a rare bug when debug session in Intellij Idea failed and hang. It was not possible to terminate it without reopening the project.

Root case

Issue was reproduced while by having broken deployment to simulator step that allowed to investigate and locate the root case.
The root case – Idea hangs on waiting for stdout/stderr to be closed on terminated (due fail to deploy to sim) process.
Root case 1: the way stdout was redirected:

stdout -> FIFO FILE -> input stream

Root case 2: wrapper process class ProcessProxy around launcher process that was never returned stream to Idea but was never closing it (here it hangs).

The fix: launcher classes rework

  • Currently, there are three launchers: ios simulator, ios device, console. All use own launcher. The fix introduced common functionality as separated AbstractLauncherProcess class. Specific implementations just extends it;
  • FIFO and redirect logic massively simplified. No FIFO, just simple Piped streams. Added observable stream to allow a plugin to monitor stdout (as part of FIFO replacement) - required by JUNIT plugin;

Console target fixes

  • fixed not working debugging of console target;
  • added support for STDIN pipe to console target;
  • console template fixed to include required <target>console</target>.

Code

PR481

AltPods: pods updated - 1.7.0-SNAPSHOT

|

AltPods were update to v1.7.0-SNAPSHOTS to sync with recent releases. Part of list didn’t receive any API update hovewer bindings were re-generated against recent version of frameworks. Update list look as bellow:

New framework

Updated pods

Unchanged pods

These pods were pushed to https://oss.sonatype.org/content/repositories/snapshots maven repo under 1.7.0-SNAPSHOTS version. Source code @github

Updates are not fullty tested, please open issue if bug found.

NB: AltPods – are robopods kept at my personal standalone repo. This is done to prevent main robopods repo from turning into code graveyard. As these pods have low interest of community and low chances to be updated.

Debugger: maintenance 2020.4

|

PR475 delivers an amount of fixes related to the debugger:

fixed: debugger crashes when setting breakpoints in specific method

Root case is changes in previous improvement: debugger reports now only line number where breakpoint can be hit.
Improvement was to re-use breakpoint map structure as lines available for breakpoints. Debugger resolves that map when line map is being requested.
If all lines in the method are available for breakpoints and number of lines is even to eight this results in available for breakpoints map completely filled with zeroes. Linker puts zero filled memory structures to __bss data segment. Debugger was not expecting symbols in this segment and was crashing with unable to resolve symbol exception.

The fix: refactored buffer reader and added ZeroReader to support .bss data segment.

fixed: broken stepping over line that loads inner class

this happens when stepping over code similar to bellow:

inner class Test2()
fun test() {
    print(1)
    Test2()    // stepping over 
    print(2)   // will not stop here first time     
}

Root case: is class loaded event from device/simulator that suspends thread. Once thread is suspended all stepping constraints removed. Debugger after receiving class loaded even tells the target to resume. But there is no stepping constraint available anymore.
The fix: restore stepping constraint in device on thread resume if stepping is active.

fixed: stepping inside kotlin lambda

The issue can be observed in code similar to bellow:

      fun test() {
/*7*/    list.forEachIndexed { index, str ->
/*8*/       text = str + index       // <-- will not step or stop at this line 
/*9*/    }
     }

Kotlin injects collections code to user class under extended line number range and uses SMAP to map extended range into class line number range. Example above turned into following pseudo code after SMAP un-mapping:

    fun test() {
/*7*/   val l = list as Iterable<String>; var idx = 0;
/*8*/   val it = l.iterator(); while(it.hasNext()) { val str = it.next(); text = str + idx++ }
    }

As result there is multiple operation an line 8. but only one breakpoint/stop point will be instrumented before first operation in line. This results in breakpoint/stop point injected before l.iterator() and no one inside the loop.

The fix: inject stop/breakpoint hooks before SMAP line numbers unmapped and code get collapsed in single line.

optimizations: application binary stripped

Binary was not stripped before as strip removes local symbols as result debugger was not able to resolve required symbols. To support stripping its requires to pre-save symbols debugger depends on. This was done by putting these into exported symbols list.
Having binary stripped results in following (sample data based on empty project, x86_64 simualator):

  • application binary size reduced: from 113M -> 45M, faster deployment;
  • number of symbols debugger have to parse reduced: from 317K -> 28K, faster parsing(10x time) and debugger startup, smaller memory footprint as no need to keep never used ~300K symbols (about 60M);

rework: refactored buffer reader

Code to manipulate memory buffer refactored to provide abstract interface instead of ByteBuffer centric implementation. Code refactored to use interfaces that allows to provide implementation with different data source. For example NullReader to simulate reading from .bss segment. Also, it allows to have future improvements required for windows builds (workaround for mapped file issues).

rework: .spfpoffset value fetched during compilation (was runtime)

Initially it was mistake to make .spfpoffset symbols global as it resulted in many “symbol already defines” error.
Root case of it is LLVM emitter who adds .spfpoffset for all functions (even always inline or private). As result there was many private symbols with same name turned into global.
Situation was fixed (for having binary stip sake) by reworking the way .spfpoffset retrieved:
was:

  • symbols were referenced in llvm.used just to survive dead code elimination;
  • debugger resolved .spfpoffset runtime and fetched value;

now:

  • during compilation .spfpoffset is being fetched from object file;
  • value added to class debug information;
  • .spfpoffset is not required anymore and can be eliminated as not used.

other changes: dependency order changed

Debugger module reworked to be independed from LLVM/Compiler module. Now its stand-alone module. Instead following was changed:

  • compiler module has direct dependency to debugger;
  • idea/eclipse/maven/gradle plugins don’t depend anymore on debugger (as it is part of compiler now);

Support for swift dependencies for static libs/frameworks

|

RoboVm compiler is able to detect swift usage in dynamic frameworks and copies all required swift libraries into application bundle.
Same logic (calling otool -L) doesn’t work for static libraries (.a) and static frameworks.
While xcframework and better logic (instead of otool -L) is still in development workaround was introduced to unblock community:

swift libraries have to be manually listed in robovm.xml:

Example:

<libs>
    <lib>libswiftCore.dylib</lib>
    <lib>libswiftCore.dylib</lib>
    <lib>libswiftCoreFoundation.dylib</lib>
    <lib>libswiftCoreGraphics.dylib</lib>
    <lib>libswiftCoreImage.dylib</lib>
    <lib>libswiftDarwin.dylib</lib>
    <lib>libswiftDispatch.dylib</lib>
    <lib>libswiftFoundation.dylib</lib>
    <lib>libswiftMetal.dylib</lib>
    <lib>libswiftObjectiveC.dylib</lib>
    <lib>libswiftQuartzCore.dylib</lib>
    <lib>libswiftUIKit.dylib</lib>
</libs>

Simplest way to find out list of required swift libraries is to use otool -l libOne.a | grep "\-lswift".

Now compiler detects swift library in dependencies and applies logic similar to dynamic frameworks:

  • adds swift library path to library search path;
  • link binary with these libraries;
  • copies swift libraries into application bundle;

Code was delivered as PR474.

AltPods: pods updated - 1.6.0-SNAPSHOT

|

AltPods were update to v1.6.0-SNAPSHOTS to sync with recent releases. Part of list didn’t receive any API update hovewer bindings were re-generated against recent version of frameworks. Update list look as bellow:

New framework

Updated pods

Unchanged pods

These pods were pushed to https://oss.sonatype.org/content/repositories/snapshots maven repo under 1.6.0-SNAPSHOTS version. Source code @github

Updates are not fullty tested, please open issue if bug found.

NB: AltPods – are robopods kept at my personal standalone repo. This is done to prevent main robopods repo from turning into code graveyard. As these pods have low interest of community and low chances to be updated.

iOS 13.4 bindings preview (beta2)

|

iOS 13.4

bindings have arrived as PR458.
selectively updated frameworks based on api diff
binding preview based on Xcode11.4 BETA 2, following was updated:

  • AuthenticationServices
  • AutomaticAssessmentConfiguration (Added)
  • AVFoundation
  • CallKit
  • CarPlay
  • ClassKit
  • Contacts
  • CoreBluetooth
  • CoreGraphics
  • CoreLocation
  • CoreMedia
  • CoreText
  • GameKit
  • ImageCaptureCore
  • Intents
  • LocalAuthentication
  • MetalPerformanceShaders (also added missing classes from previous iOS)
  • NetworkExtension
  • PassKit
  • StoreKit
  • UIKit
  • WebKit

Framework improved -- exposing ObjectiveC classes to natively create instances

|

Previous improvement simplified way Framework target was created right out the box without need to write native code. But it didn’t allow creating objects native way and workaround was to use single point singleton and consider objects as protocols. RoboVM creates objects runtime using objc/runtime API so there is no information available during linkage phase. And a try to use it will cause symbol not found exception like this:

Undefined symbols for architecture x8664: “_OBJC_CLASS$_RoboClass”, referenced from: objc-class-ref in ViewController.o

The goal of this Improvement:

  • hide any need to initialize JVM from user;
  • allow natively creating and using of java classes as native.

In general it should be enough:

  • drop framework in Xcode project;
  • instantiate and use classes like bellow:
    let demo = MyFrameworkDemo(text: "demo")!
    print(demo.roboVmVersion()!)
    // or
    MyFrameworkDemo *demo = [[MyFrameworkDemo alloc] initWithText:@"demo"];
    NSLog(@"%@", [demo roboVmVersion]);
    

(skip long read and start creating framework)

Step one. OBJC_CLASS_$_${CLASSNAME} structure

Idea plugin: run configuration maintenance

|

Why

Idea run configuration might become invalid if configured items become invalid, like:

  • simulator is not available in next xcode;
  • signing identity or provision profile expired or removed;

Configuration editor was hiding errors

In this case compilation failed with runtime exception. But once run configuration editor is opened instead of pointing to invalid items editor was auto picking up first one available. As result user was not able to see a problem and to have configuration overridden changes to it to be done. Otherwise apply button was not active.

Name instead unique id was used as identifier

To identify signing identity/simulator/provisioning profile its name was saved. For auto value just auto text placeholder was saved and checked against. This is subject for name collisions.

Whats changed

Auto mode explicitly specified

Field that might be auto (Signing identity/Provisioning profile/Simulator) are now have separate parameter in config file that specifies entry type: auto or explicit ID. And auto mode is identified by it and not string constant in entry name.

Explicit identifier instead of name

When explicit entry is used (e.g. not auto) entries id is saved instead of entry name (udid for profile/simulator, footprint for signing identity). No collision anymore.

Auto mode for simulator

Simulator field receives two modes auto iPhone and auto iPad. These modes acts similarly to gradle’s one ‘launchIPhoneSimulator’/launchIPadSimulator. Also in case there is no simulator available in system it will allows to recover broken run configuration.

Quick fix for broken fields

Problem fields can be quick fixed to auto value with Fix button.

arm64 is default for device target

Default thumbv7 was a problem as is not available since ios11 (on most devices today). This caused deployment error at very end of run cycle. Having arm64 default improves user experience.

no more x86 (32bit) option for ios11+ simulators

Similar issue to arm64. It 32bit code doesn’t run on ios11+ simulators.

no more don't sign options

don't sign option was for usage with jailbroken ios devices. While having free provisioning from Apple this option have no more sense. Also ldid binary was removed. If don't sign option is specified from gradle/maven ADHOC signing will be performed.

bonus1: gradle import of template projects was fixed

Created from template gradle projects were broken due:

  • gradle was set to use RoboVM SDK (and it is not operational any more for tools);
  • Java source code level was set to Java12+;

Now: gradle uses internal JDK, java source level forced to Java8.

bonus2: added support for Apple Development certificates

These were not part of regex and were ignored when auto identity was specified.

bonus3: suppressed -559038737 error code

If simulator process is terminated by Idea strange error message was printed into console log:

Process finished with exit code -559038737

This code is UNKNOWN_CODE constant in Executor. Now it is being recognized and replaced with 0.

Code

Code was delivered as PR456