get rid of launcher script requirement; fix app data dir being hardcoded; update documentation; remove prebuilt dalvik (NOTE - purged from history at this point, so is not part of the diff)

README.md

@@ -3,19 +3,41 @@
 ---
 
 build instructions:  
-`make`: compile everything (except dalvik, which is precompiled from https://gitlab.com/Mis012/dalvik_standalone)  
+1. compile and install https://gitlab.com/Mis012/dalvik_standalone (art branch) (you can run it from builddir, but it requires some additional env variables)
+2. `mkdir builddir`
+3. `meson builddir`
+4. `ninja -C builddir`
 
-usage:  
+then, to run from builddir:  
-`./launch_activity.sh <path_to_apk> <path_to_activity>`  
+`cd builddir`  
-example: `./launch_activity.sh test_apks/org.happysanta.gd_29.apk org/happysanta/gd/GDActivity`  
+and  
-note: some apps don't like runtime changes to resolution, and currently GLSurfaceView will stretch instead of changing resolution  
+`RUN_FROM_BUILDDIR= LD_LIBRARY_PATH=./ ANDROID_APP_DATA_DIR=../test_apks/example_data_dir/ ./android_translation_layer ../test_apks/org.happysanta.gd_29.apk -l org/happysanta/gd/GDActivity`  
-example with custom width/height: `./launch_activity.sh test_apks/org.happysanta.gd_29.apk 'org/happysanta/gd/GDActivity -w 540 -h 960'`
+(for an example of a full game working that can be distributed along this)
+or  
+`RUN_FROM_BUILDDIR= LD_LIBRARY_PATH=./ ANDROID_APP_DATA_DIR=../test_apks/example_data_dir/ ./android_translation_layer ../test_apks/gles3jni.apk -l com/android/gles3jni/GLES3JNIActivity`  
+(for a sample app using OpenGL from native code to do it's rendering) (note: the lib in `test_apks/example_data_dir/gles3jni.apk/lib/` is for x86_64, make sure you extract the lib for your architecture from the apk)  
 
-NOTE: you might need to copy some files out from the apk under `data/`, e.g the `assets` folder; 
+to install:  
+`meson install`  
+
+to run after installataion:  
+`ANDROID_APP_DATA_DIR=[data dir] android_translation_layer [path to apk] -l [activity to launch]`  
+or just  
+`android_translation_layer [path to apk] -l [activity to launch]`  
+to use the default data dir of `~/.local/share/android_translation_layer/`
+
+NOTE: some apps don't like runtime changes to resolution, and currently GLSurfaceView will stretch instead of changing resolution  
+to sidestep this, we allow for specifying the initial resolution, which will currently always get passed as the screen resolution to the app and to GLSurfaceView even when you resize the window.
+example with custom width/height: `android_translation_layer path/to/org.happysanta.gd_29.apk -l org/happysanta/gd/GDActivity -w 540 -h 960`
+
+NOTE: you might need to copy some files out from the apk under `ANDROID_APP_DATA_DIR` 
+(defaults to `~/.local/share/android_translation_layer/`), e.g the `assets` folder;  
 additionally, resources (`res`) currently need to be "decompiled" (e.g. by apktool, though this 
-additionally replaces hex IDs with string names, which needs to be reversed)  
+additionally replaces hex IDs with string names, which then needs to be manually reversed;
+android studio's `inspect apk` feature is known to keep the integers)  
 NOTE: on X11, Gtk might decide to use GLX, which completely messes up our EGL-dependent code.
 If you have a debug build of Gtk, you can use GDK_DEBUG=gl-egl to force the use of EGL  
+NOTE: we don't currently handle signed apks; simply remove the META-INF folder from an apk to skip signature verification
 
 when it doesn't work:  
 if you are trying to launch a random app, chances are that we are missing implementations for some  
@@ -38,8 +60,6 @@ screenshot:
 
 ![angry birds 3.2.0, Worms 2 Armageddon, and gravity defied running side by side by side](https://gitlab.com/Mis012/android_translation_layer_PoC/-/raw/master/screenshot_2.png)
 
-note: running two or more different apps at the same time needs a tiny bit of luck (no conflicting files), since the data directory path is currently hardcoded to `./data/` no matter the app  
-
 ##### FAQ:
 
 Q:  
@@ -67,25 +87,17 @@ A:
 	well, first things first, technically I compiled Gravity Defied myself and removed  
 	some bug-reporting-init-related code which I got frustrated with stubbing out. however,  
 	adding more stubs should make that unnecessary.  
-	now for the second issue: `./data/` contains some stuff that should instead be read from the apk,  
+	now for the second issue: `ANDROID_APP_DATA_DIR` contains some stuff that should instead be  
-	and some of this stuff is also externally converted (e.g Xml from binary form to actual Xml).  
+	read from the apk, and some of this stuff is also externally converted (e.g Xml from binary  
-	obviously this is not ideal for user experience. NOTE: it seems that the binary form *might* be  
+	form to actual Xml). obviously this is not ideal for user experience.  
-	protobuf-based, which would make reading it directly easier.  
+	NOTE: it seems that the binary form *might* be protobuf-based, which would make reading it  
+	directly easier.  
 	and the third issue: Gravity Defied is still extremely simple compared to most android apps,  
 	doesn't acknowledge compat layers, and the most intricate UI element is completely custom drawn  
 	using the canvas interface, in a manner that makes it easy to implement with cairo.  
 	angry birds (old version) and worms 2 armageddon were chosen because they similarly don't use  
 	compat layers, and basically the entire UI is custom drawn with OpenGL calls from native code.
 
-Q:  
-	why are only 32bit architectures supported?
-A:  
-	we are currently using good old dalvik vm (probably newer commit than ever shipped though), which  
-	is so hopelessly hardcoded to assume 32bit ints and pointers that it probably made google's cost/benefit  
-	analyses of writing art from scratch a lot easier.  
-	Eventually, we should probably move to ART, but it's an open question whether that will bring issues for our  
-	usecase (is there a way to use ART without zygote? is it straightforward to port the few modifications that we did to dalvik?)
-
 
 ##### Roadmap:
 
@@ -99,6 +111,4 @@ A:
 
 - especially implement the alternatives to GLSurfaceView (using SurfaceView to get an EGL surface, native activity, not sure if there are others?) which would allow us to support a few more 99%-native applications with relative ease
 
-- explore switching to ART in the interest of supporting 64bit architectures
+- explore using bubblewrap to enforce the security policies that google helpfully forces apps to comply with (and our own security policies, like no internet access for apps which really shouldn't need it and are not scummy enough to refuse to launch without it)
-
-- explore using apparmor to enforce the security policies that google helpfully forces apps to comply with (and our own security policies, like no internet access for apps which really shouldn't need it and are not scummy enough to refuse to launch without it)

doc/Architecture.md

@@ -1,25 +1,14 @@
 #### directory structure
-
+`src/arsc_parser/` - Java .arsc parser I found somewhere, with fixes (should eventually get replaced by C code)
-`dalvik/*dalvik` - helper scripts for running bytecode in the dalvik VM (not used anymore, we link against `libdvm.so` directly now)
-`dalvik/linux-${arch}/` - pre-complied dalvik from https://gitlab.com/Mis012/dalvik_standalone
-
-`arsc_parser/` - Java .arsc parser I found somewhere, with fixes (should eventually get replaced by C code)
-`data/` - the equivalent of `/data/data/${app-package-name}/`; TODO - use `data/${app-package-name}` instead 
-to allow for storing the data of multiple apps at the same time
-`data/lib/` - hardcoded location, libraries under which are assumed to be linked against bionic 
-(and will therefore be loaded with a shim bionic linker)
 `doc/` - documentation
-`jars/` - when we want to link against dalvik core java libs, we need to keep their non-dex versions here
+`jars/` - contains core-libart-hostdex_classes.jar which we use as compile-time bootclasspath (TODO: have art-dev package install this system-wide)
-`jni/` - C code implementing things which it doesn't make sense to do in Java (ideally this would be most things)
+`src/api-impl/` - Java code implementing the android APIs
-`libandroid-src/` - C code implementing `libandroid.so` (this is needed by most JNI libs which come with android apps)
+`src/api-impl-jni/` - C code implementing things which it doesn't make sense to do in Java (ideally this would be most things)
-`libnative/` - compilation output for our `.so` libraries (currently `libtranslation_layer_main.so` and `libandroid.so`)
+`src/libandroid/` - C code implementing `libandroid.so` (this is needed by most JNI libs which come with android apps)
-`main-src/` - Java code implementing the android APIs
+`src/main-executable` - code for the main executable, which sets stuff up (including the JVM) and launches the activity specified on the cmdline
-`src/` - code for the main executable, which sets stuff up (including the JVM) and launches the activity specified on the cmdline
 `test-apks/` - all apks known to somewhat work, except paid proprietary ones which we sadly can't really include
 
 `./com.google.android.gms.apk`: microg; stopgap solution to run apps with GSF dependencies (fwiw, the actual solution will likely also use MicroG)
-`./*.dex` - compiled Java code; `hax_arsc_parser.dex` corresponds to `hax_arsc_parser.dex/`,
-`hax.dex` corresponds to `main-src`
 
 #### philosophy
 
@@ -38,28 +27,22 @@ components, we also keep just the respective .so files from those apps
 and provide implementations/shims for the system libraries they are linked
 against)
 
-#### current control flow (to be refined)
+#### current control flow
 
-1. the user executes `launch_activity.sh`  
+1. the user executes the main executable (`android_translation_layer`)  
-this wrapper accomplishes the following things:
-	- `LD_PRELOAD` makes sure that `libbsd` (which provides some functions that bionic would on android) is loaded, even though bionic-linked libraries don't know they want it
-	- `LD_PRELOAD` also loads libpthread_bio.so, which is a shim for translating libpthread calls made by bionic-linked libraries to calls to glibc/musl libpthread
-	- `ANDROID_ROOT` and `ANDROID_DATA` get set, both of which are required by dalvik
-	- `BIONIC_LD_LIBRARY_PATH` gets set to the path to the android app's lib folder (currently any app's data directory is `data/`) so that the shim bionic linker knows which .so files must not be loaded by the system linker
-	- `LD_LIBRARY_PATH` is set up such that both the host linker and the shim bionic linker can find any non-system libraries they will need
-	- `./main ${1} -l ${2}` calls the main executable and instructs it to load the apk file `${1}` into classpath before launching the activity `${2}` using the JVM
 
-2. the executable is compiled from `src/main.c`:
+2. the executable is compiled from `src/main-executable/main.c`:
 	1. `int main(int argc, char **argv)` sets up a GtkApllication, cmdline handling, and calls `g_application_run`
 	2. GtkApplication glue parses the cmdline and calls `static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* hint, struct jni_callback_data *d)`
 	3. `static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* hint, struct jni_callback_data *d)`:
 		1. constructs the classpath from the following:
-			- `hax_arsc_parser.dex` is dalvik bytecode implementing .arsc parsing duties (to be replaced by C code eventually)
+			- the path to api-impl.jar (contains the following, renamed to classes{2}.dex so that art loads them)
-			- `hax.dex` contains all the implementations of android framework functions
+				- `hax_arsc_parser.dex` is dalvik bytecode implementing .arsc parsing duties (to be replaced by C code eventually)
-			- `%s` will be substituted by the path to the app's apk (passed to us on cmdline), making the bytecode within (and resources.arsc, which is currently the only other file read straight from the apk) available in classpath
+				- `hax.dex` contains all the implementations of android framework functions
-			- `com.google.android.gms.apk` is the path to a microG apk, needed for apps with a dependency on GSF; this is specified after the app's apk so that the the app's apk is the first zip file in the classpath (needed for getting the right resources.arsc, TODO: ask for the classloader which loaded the activity that was specified on the cmdline)
+			- the path to the app's apk (passed to us on cmdline), making the bytecode within (and resources.arsc, which is currently the only other file read straight from the apk) available in classpath
-		2. contructs other options for and launches the dalvik virtual machine
+			- the path to a microG apk, needed for apps with a dependency on GSF; this is specified after the app's apk so that the the app's apk is the first zip file in the classpath (needed for getting the right resources.arsc, TODO: ask for the classloader which loaded the activity that was specified on the cmdline)
-		3. load `libtrasnlation_layer_main.so` using the internal function `dvmLoadNativeCode` (this contains our native methods, and dalvik will only search in libraries that were loaded using this or System.load) (NOTE: care is taken to register this with the right classloader so the java code which declares the native methods is the java code that can use them)
+		2. contructs other options (mainly library path) for and launches the dalvik virtual machine
+		3. load `libtrasnlation_layer_main.so` using the internal version of Runtime.loadLibrary which lets us register this with the right classloader so the java code which declares the native methods is the java code that can use them
 		4. sets up a JNI handle cache
 		5. sets display size to be passed to apps according to optionally specified window size (some apps handle runtime resizing poorly, and our EGL code currently doesn't handle it at all)
 		6. creates a Gtk Window with said size and shows it on screen
@@ -68,6 +51,6 @@ this wrapper accomplishes the following things:
 3. the Activity specfified with `-l` calls various android APIs, which we hopefully implement; 
 typically, it will set up some android widgets, which will appear in the Gtk window as Gtk widgets
 
-5. you, the user, interact with the Gtk widgets, which the app might have registered callbacks for; 
+4. you, the user, interact with the Gtk widgets, which the app might have registered callbacks for; 
 as long as our implementation of android APIs is good enough for what the app needs, you can use the app
 as you would on android

doc/QuickHelp.md

@@ -11,7 +11,7 @@ you would probably prefer to first get the app to launch without errors. With a
 with writing stubs.
 
 What is a stub? Well, the simplest stub would be an empty class, like this:
-`main-src/android/webkit/WebView.java:`
+`src/api-impl/android/webkit/WebView.java:`
 ```Java
 package android.webkit;
 
@@ -34,7 +34,7 @@ public class WebView {
 	}
 }
 ```
-here, all that you need to take care of is that at `main-src/android/content/Context.java`, you have at minimum 
+here, all that you need to take care of is that at `src/api-impl/android/content/Context.java`, you have at minimum 
 a stub class.
 
 Unfortunately, in the WebView case, the method that an app was trying call wasn't returning `void`. If this is 
@@ -94,7 +94,7 @@ to get to work in order to cut down on the amount of stubbing you need to do.
 There are two basic types of widgets (Views): containers (Layouts) and the rest.
 
 To implement a container widget, simply copy an existing container widget implementation (e.g LinearLayout 
-(`./main-src/android/widget/LinearLayout.java` and `./jni/widgets/android_widget_LinearLayout.c`)), and that's 
+(`src/api-impl/android/widget/LinearLayout.java` and `src/api-impl-jni/widgets/android_widget_LinearLayout.c`)), and that's 
 it! Now, chances are that you wanted something slightly different, but this will at least display the child 
 widgets so that you can focus on implementing those.
 
@@ -108,7 +108,7 @@ the close-enough Gtk widget, since subclassing is mostly not possible in Gtk.
 
 ###### case study: ImageView
 
-`main-src/android/widget/ImageView.java`
+`src/api-impl/android/widget/ImageView.java`
 ```Java
 package android.widget;
 ```
@@ -151,7 +151,7 @@ public class ImageView extends View {
 
 ---
 
-`jni/widgets/android_widget_ImageView.c`
+`src/api-impl-jni/widgets/android_widget_ImageView.c`
 ```C
 #include <gtk/gtk.h>
 
@@ -161,11 +161,11 @@ public class ImageView extends View {
 #include "WrapperWidget.h"
 
 ```
-↑ every widget will be under `jni/widgets/` and will have these includes
+↑ every widget will be under `src/api-impl-jni/widgets/` and will have these includes
 ```C
-#include "android_widget_ImageView.h"
+#include "../generated_headers/android_widget_ImageView.h"
 ```
-↑ this is the jni-generated header file; you will need to add this to the part of the Makefile which moves widget header files to `jni/widgets/`
+↑ this is the jni-generated header file (btw, t's name is what dicates the name of this .c file)
 ↓ there should be two functions here, one for the `Context` costructor and one for the `AttributeSet` one; for start, you can keep them the same
 ```C
 JNIEXPORT void JNICALL Java_android_widget_ImageView_native_1constructor__Landroid_content_Context_2(JNIEnv *env, jobject this, jobject context)
@@ -173,7 +173,7 @@ JNIEXPORT void JNICALL Java_android_widget_ImageView_native_1constructor__Landro
 	GtkWidget *wrapper = wrapper_widget_new();
 ```
 ↑ the wrapper widget is required, it's expected by generic functions operating on widgets; the purpose is to allow for things like background image
-handling independently of the
+handling for cases where we can't subclass the backing widget itself
 ```C
 	GtkWidget *image = gtk_image_new_from_icon_name("FIXME"); // will not actually use gtk_image_new_from_icon_name when implementing this, but we want that nice "broken image" icon
 ```

meson.build

@@ -6,74 +6,80 @@ add_project_dependencies(incdir_dep, language: 'c')
 cc = meson.get_compiler('c')
 dir_base = meson.current_source_dir()
 builddir_base = meson.current_build_dir()
+# FIXME: make art install a pkgconfig file
 libart_dep = [
-	cc.find_library('art', dirs : [join_paths(dir_base, 'dalvik/linux-x86/lib64/')]),
+	cc.find_library('art', dirs : [ '/usr' / get_option('libdir') / 'art', '/usr/local' / get_option('libdir') / 'art', get_option('prefix') / get_option('libdir') / 'art' ]),
-     	cc.find_library('nativebridge', dirs : [join_paths(dir_base, 'dalvik/linux-x86/lib64/')])
+     	cc.find_library('nativebridge', dirs : [ '/usr' / get_option('libdir') / 'art', '/usr/local' / get_option('libdir') / 'art', get_option('prefix') / get_option('libdir') / 'art' ])
 ]
 libdl_bio_dep = [
-	cc.find_library('dl_bio', dirs : [join_paths(dir_base, 'dalvik/linux-x86/lib64/')])
+	cc.find_library('dl_bio')
 ]
 
 libtranslationlayer_so = shared_library('translation_layer_main', [
-                                                             	'src/api-impl-jni/egl/com_google_android_gles_jni_EGLImpl.c',
+                                                                  	'src/api-impl-jni/egl/com_google_android_gles_jni_EGLImpl.c',
-                                                             	'src/api-impl-jni/android_os_SystemClock.c',
+                                                                  	'src/api-impl-jni/android_os_Environment.c',
-                                                             	'src/api-impl-jni/android_view_Window.c',
+                                                                  	'src/api-impl-jni/android_os_SystemClock.c',
-                                                             	'src/api-impl-jni/util.c',
+                                                                  	'src/api-impl-jni/android_view_Window.c',
-                                                             	'src/api-impl-jni/android_graphics_Canvas.c',
+                                                                  	'src/api-impl-jni/util.c',
-                                                             	'src/api-impl-jni/drawables/ninepatch.c',
+                                                                  	'src/api-impl-jni/android_graphics_Canvas.c',
-                                                             	'src/api-impl-jni/android_content_res_AssetManager.c',
+                                                                  	'src/api-impl-jni/drawables/ninepatch.c',
-                                                             	'src/api-impl-jni/audio/android_media_AudioTrack.c',
+                                                                  	'src/api-impl-jni/android_content_res_AssetManager.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_RelativeLayout.c',
+                                                                  	'src/api-impl-jni/audio/android_media_AudioTrack.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_ScrollView.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_RelativeLayout.c',
-                                                             	'src/api-impl-jni/widgets/android_opengl_GLSurfaceView.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_ScrollView.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_ImageView.c',
+                                                                  	'src/api-impl-jni/widgets/android_opengl_GLSurfaceView.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_FrameLayout.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_ImageView.c',
-                                                             	'src/api-impl-jni/widgets/WrapperWidget.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_FrameLayout.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_TextView.c',
+                                                                  	'src/api-impl-jni/widgets/WrapperWidget.c',
-                                                             	'src/api-impl-jni/widgets/android_widget_LinearLayout.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_TextView.c',
-                                                             	'src/api-impl-jni/views/android_view_View.c',
+                                                                  	'src/api-impl-jni/widgets/android_widget_LinearLayout.c',
-                                                             	'src/api-impl-jni/views/android_view_ViewGroup.c',
+                                                                  	'src/api-impl-jni/views/android_view_View.c',
-                                                             	'src/api-impl-jni/android_graphics_Bitmap.c'                                                             ],
+                                                                  	'src/api-impl-jni/views/android_view_ViewGroup.c',
-                                                             dependencies: [
+                                                                  	'src/api-impl-jni/android_graphics_Bitmap.c'
-                                                             	dependency('gtk4'), dependency('gl'), dependency('egl'), dependency('jni')
+                                                                  ],
-                                                             ],
+                                                                  install: true,
-                                                             link_args: [
+                                                                  install_dir : get_option('libdir') / 'java/dex/android_translation_layer/natives',
-                                                             	'-lasound'
+                                                                  dependencies: [
-                                                             ])
+                                                                  	dependency('gtk4'), dependency('gl'), dependency('egl'), dependency('jni')
+                                                                  ],
+                                                                  link_args: [
+                                                                  	'-lasound'
+                                                                  ])
 
-conf_data = configuration_data()
+executable('android-translation-layer', [ 
-conf_data.set('install_libdir', get_option('prefix') / get_option('libdir'))
-configure_file(input : 'src/config.h.in',
-               output : 'config.h',
-               configuration : conf_data)
-
-configure_file(input : 'launch_activity.sh.in',
-               output : 'launch_activity.sh',
-               configuration : conf_data)
-
-executable('main', [ 
                    	'src/main-executable/main.c',
                    	'src/main-executable/r_debug.c'
                    ],
+                   install: true,
                    dependencies: [
                    	dependency('gtk4'), dependency('jni'), declare_dependency(link_with: libtranslationlayer_so), libart_dep, dependency('dl'), libdl_bio_dep
+                   ],
+                   link_args: [
+                   	'-rdynamic'
                    ])
 
 # libandroid
 shared_library('android', [
-              	'src/libandroid/misc.c',
+                          	'src/libandroid/misc.c',
-              	'src/libandroid/asset_manager.c'
+                          	'src/libandroid/asset_manager.c'
-              ])
+                          ],
+                          install: true,
+                          soversion: 0,)
 
 # hax_arsc_parser.dex (named as classes2.dex so it works inside a jar)
 subdir('src/arsc_parser')
-hax_arsc_parser_dex = custom_target('hax_arsc_parser.dex', build_by_default: true, input: [hax_arsc_parser_jar], output: ['classes2.dex'], command: [join_paths(dir_base, 'dalvik/linux-x86/bin/dx'),'--verbose', '--dex', '--output='+join_paths(builddir_base, 'classes2.dex'), hax_arsc_parser_jar.full_path()])
+hax_arsc_parser_dex = custom_target('hax_arsc_parser.dex', build_by_default: true, input: [hax_arsc_parser_jar], output: ['classes2.dex'],
+                                    command: ['dx', '--verbose', '--dex', '--output='+join_paths(builddir_base, 'classes2.dex'), hax_arsc_parser_jar.full_path()])
 
 # hax.dex (named as classes.dex so it works inside a jar)
 subdir('src/api-impl')
-hax_dex = custom_target('hax.dex', build_by_default: true, input: [hax_jar], output: ['classes.dex'], command: [join_paths(dir_base, 'dalvik/linux-x86/bin/dx'),'--verbose', '--dex', '--output='+join_paths(builddir_base, 'classes.dex'), hax_jar.full_path()])
+hax_dex = custom_target('hax.dex', build_by_default: true, input: [hax_jar], output: ['classes.dex'],
+                        command: ['dx', '--verbose', '--dex', '--output='+join_paths(builddir_base, 'classes.dex'), hax_jar.full_path()])
 
 # api-impl.jar
-custom_target('api-impl.jar', build_by_default: true, input: [hax_dex, hax_arsc_parser_dex], output: ['api-impl.jar'], command: ['zip', '-j', join_paths(builddir_base, 'api-impl.jar'), hax_dex.full_path(), hax_arsc_parser_dex.full_path()])
+custom_target('api-impl.jar', build_by_default: true, input: [hax_dex, hax_arsc_parser_dex], output: ['api-impl.jar'],
+              install: true,
+              install_dir : get_option('libdir') / 'java/dex/android_translation_layer',
+              command: ['jar', '-cvf', join_paths(builddir_base, 'api-impl.jar'), '-C', builddir_base, hax_dex, '-C', builddir_base, hax_arsc_parser_dex])
 
 

src/api-impl-jni/android_content_res_AssetManager.c

@@ -9,15 +9,18 @@
 #include "util.h"
 #include "generated_headers/android_content_res_AssetManager.h"
 
-#define ASSET_DIR "data/assets/"
+#define ASSET_DIR "assets/"
+char *get_app_data_dir();
 
 JNIEXPORT jint JNICALL Java_android_content_res_AssetManager_openAsset(JNIEnv *env, jobject this, jstring _file_name, jint mode)
 {
 	const char *file_name = _CSTRING(_file_name);
-	char *path = malloc(strlen(file_name) + strlen(ASSET_DIR) + 1);
+	char *app_data_dir = get_app_data_dir();
+	char *path = malloc(strlen(app_data_dir) + strlen(ASSET_DIR) + strlen(file_name) + 1);
 	int fd;
 
-	strcpy(path, ASSET_DIR);
+	strcpy(path, app_data_dir);
+	strcat(path, ASSET_DIR);
 	strcat(path, file_name);
 
 	printf("openning asset with filename: %s\n", _CSTRING(_file_name));

src/api-impl-jni/android_os_Environment.c

@@ -0,0 +1,11 @@
+#include "defines.h"
+#include "util.h"
+
+#include "generated_headers/android_os_Environment.h"
+
+char *get_app_data_dir();
+
+JNIEXPORT jstring JNICALL Java_android_os_Environment_native_1get_1app_1data_1dir(JNIEnv *env, jclass this)
+{
+	return _JSTRING(get_app_data_dir());
+}

src/api-impl-jni/defines.h

@@ -13,6 +13,7 @@
 #define _CLASS(object) ((*env)->GetObjectClass(env, object))
 #define _SUPER(object) ((*env)->GetSuperclass(env, object))
 #define _METHOD(class, method, attrs) ((*env)->GetMethodID(env, class, method, attrs))
+#define _STATIC_METHOD(class, method, attrs) ((*env)->GetStaticMethodID(env, class, method, attrs))
 #define _JSTRING(cstring) ((*env)->NewStringUTF(env, cstring))
 #define _CSTRING(jstring) ((*env)->GetStringUTFChars(env, jstring, NULL))
 #define _FIELD_ID(class, field, type) ((*env)->GetFieldID(env, class , field, type))

src/api-impl-jni/generated_headers/android_os_Environment.h

@@ -0,0 +1,21 @@
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include <jni.h>
+/* Header for class android_os_Environment */
+
+#ifndef _Included_android_os_Environment
+#define _Included_android_os_Environment
+#ifdef __cplusplus
+extern "C" {
+#endif
+/*
+ * Class:     android_os_Environment
+ * Method:    native_get_app_data_dir
+ * Signature: ()Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_android_os_Environment_native_1get_1app_1data_1dir
+  (JNIEnv *, jclass);
+
+#ifdef __cplusplus
+}
+#endif
+#endif

src/api-impl/android/app/Activity.java

@@ -112,7 +112,7 @@ public class Activity extends Context {
     public void setContentView(int layoutResID) throws Exception {
 		System.out.println("- setContentView - yay!");
 
-		String layout_xml_file = "data/" + getString(layoutResID);
+		String layout_xml_file = android.os.Environment.getExternalStorageDirectory().getPath() + "/" + getString(layoutResID);
 
 		System.out.println("loading layout from: " + layout_xml_file);
 

src/api-impl/android/content/Context.java

@@ -122,7 +122,7 @@ public class Context extends Object {
 
     private File getDataDirFile() {
 		if(data_dir == null) {
-			data_dir = new File("data/");
+			data_dir = android.os.Environment.getExternalStorageDirectory();
 		}
 		return data_dir;
     }
@@ -185,7 +185,7 @@ public class Context extends Object {
 
 	public FileOutputStream openFileOutput(String name, int mode) throws java.io.FileNotFoundException {
 		System.out.println("openFileOutput called for: '"+name+"'");
-		return new FileOutputStream("data/files/" + name);
+		return new FileOutputStream(android.os.Environment.getExternalStorageDirectory().getPath() + "/files/" + name);
 	}
 
 	public int checkCallingOrSelfPermission(String permission) {

src/api-impl/android/content/res/AssetManager.java

@@ -467,7 +467,7 @@ public final class AssetManager {
 		factory.setNamespaceAware(true);
 		XmlPullParser xpp = factory.newPullParser();
 
-		xpp.setInput( new FileReader("data/" + fileName) );
+		xpp.setInput( new FileReader(android.os.Environment.getExternalStorageDirectory().getPath() + "/" + fileName) );
 
 		return (XmlResourceParser)xpp;
     }

src/api-impl/android/graphics/Bitmap.java

@@ -111,7 +111,7 @@ public final class Bitmap {
 	} // FIXME
 
 	Bitmap(String path) {
-		pixbuf = native_bitmap_from_path("data/"+path);
+		pixbuf = native_bitmap_from_path(android.os.Environment.getExternalStorageDirectory().getPath() + "/" + path);
 
 		mIsMutable = false;
 		mIsPremultiplied = false;

src/api-impl/android/view/LayoutInflater.java

@@ -91,7 +91,7 @@ public class LayoutInflater {
 
     public View inflate(int layoutResID, ViewGroup root, boolean attachToRoot) throws Exception {
 
-		String layout_xml_file = "data/" + Context.this_application.getString(layoutResID);
+		String layout_xml_file = android.os.Environment.getExternalStorageDirectory().getPath() + "/" + Context.this_application.getString(layoutResID);
 
 		System.out.println("loading layout from: " + layout_xml_file);
 

src/config.h.in

@@ -1,6 +0,0 @@
-#ifndef CONFIG_H
-#define CONFIG_H
-
-#define INSTALL_LIBDIR "@install_libdir@"
-
-#endif

src/libandroid/asset_manager.c

@@ -18,14 +18,16 @@ typedef int64_t off64_t;
 typedef void JNIEnv;
 typedef void * jobject;
 
-#define ASSET_DIR "data/assets/"
+#define ASSET_DIR "assets/"
+char *get_app_data_dir();
 
 struct AAsset* AAssetManager_open(struct AAssetManager *amgr, const char *file_name, int mode)
 {
-	char *path = malloc(strlen(file_name) + strlen(ASSET_DIR) + 1);
+	char *app_data_dir = get_app_data_dir();
+	char *path = malloc(strlen(app_data_dir) + strlen(ASSET_DIR) + strlen(file_name) + 1);
 	int fd;
-
+	strcpy(path, app_data_dir);
-	strcpy(path, ASSET_DIR);
+	strcat(path, ASSET_DIR);
 	strcat(path, file_name);
 
 	printf("openning asset with filename: %s\n", file_name);

src/main-executable/main.c

@@ -1,3 +1,6 @@
+// for dladdr
+#define _GNU_SOURCE
+
 #include <gtk/gtk.h>
 
 #include "../api-impl-jni/defines.h"
@@ -8,13 +11,6 @@
 #include <stdio.h>
 #include <sys/stat.h>
 
-// generated by meson
-#include "config.h"
-
-// TODO: arguably we shouldn't rely on our install prefix being the same as art's install prefix
-// maybe we can have art export a function returning it's install prefix for us
-#define INSTALL_DEXDIR INSTALL_LIBDIR "/java/dex/"
-
 GtkWidget *window;
 
 // standard Gtk Application stuff, more or less
@@ -28,22 +24,30 @@ gboolean app_exit(GtkWindow* self, JNIEnv *env) // TODO: do more cleanup?
 	return false;
 }
 
+// this is the equivalent of /data/data/com.example.app/
+char *app_data_dir = NULL;
+char *get_app_data_dir()
+{
+	return app_data_dir;
+}
+
 // FIXME: used by hacks in GLSurfaceView
 int FIXME__WIDTH;
 int FIXME__HEIGHT;
 
-char *construct_classpath(char *prefix, char *path_to_prepend, char **cp_array, size_t len)
+char *construct_classpath(char *prefix, char **cp_array, size_t len)
 {
 	size_t result_len = strlen(prefix);
 	for(int i = 0; i < len; i++) {
-		result_len += strlen(path_to_prepend) + strlen(cp_array[i]) + 1; // the 1 is for either : or the final \0
+		if(cp_array[i])
+			result_len += strlen(cp_array[i]) + 1; // the 1 is for either : or the final \0
 	}
 
 	char *result = malloc(result_len);
 	strcpy(result, prefix);
 	for(int i = 0; i < len; i++) {
-		strcat(result, path_to_prepend);
+		if(cp_array[i])
-		strcat(result, cp_array[i]);
+			strcat(result, cp_array[i]);
 		if (i < (len - 1))
 			strcat(result, ":");
 	}
@@ -51,28 +55,7 @@ char *construct_classpath(char *prefix, char *path_to_prepend, char **cp_array,
 	return result;
 }
 
-char *construct_boot_classpath(char *prefix, char **cp_array, size_t len)
+JNIEnv* create_vm(char *api_impl_jar, char *apk_classpath, char *microg_apk, char *api_impl_natives_dir, char *app_lib_dir) {
-{
-	char *framework_dir_path;
-	const char *android_root_path = getenv("ANDROID_ROOT");
-	if(!android_root_path) { // look in PREFIX by default
-		framework_dir_path = INSTALL_DEXDIR "/art/";
-	}
-	else { // if ANDROID_ROOT is set, look there
-		const char *framework_dir = "/framework/";
-		framework_dir_path = malloc(strlen(android_root_path) + strlen(framework_dir) + 1);
-		strcpy(framework_dir_path, android_root_path);
-		strcat(framework_dir_path, framework_dir);
-	}
-
-	char *result = construct_classpath(prefix, framework_dir_path, cp_array, len);
-	if(android_root_path) // otherwise it's not allocated dynamically
-		free(framework_dir_path);
-
-	return result;
-}
-
-JNIEnv* create_vm(char *api_impl_jar, char *apk_classpath, char *microg_apk) {
 	JavaVM* jvm;
 	JNIEnv* env;
 	JavaVMInitArgs args;
@@ -80,23 +63,16 @@ JNIEnv* create_vm(char *api_impl_jar, char *apk_classpath, char *microg_apk) {
 	args.version = JNI_VERSION_1_6;
 	args.nOptions = 3;
 
-	// TODO: should probably not hardcode this
+	if(getenv("RUN_FROM_BUILDDIR")) {
-	char *boot_cp_arr[] = {
+		options[0].optionString = construct_classpath("-Djava.library.path=", (char *[]){"./", app_lib_dir}, 2);
-		"apache-xml-hostdex.jar",
+	} else {
-		"core-junit-hostdex.jar",
+		printf(">>%s<<\n", api_impl_natives_dir);
-		"core-libart-hostdex.jar",
+		options[0].optionString = construct_classpath("-Djava.library.path=", (char *[]){api_impl_natives_dir, app_lib_dir}, 2);
-		"hamcrest-hostdex.jar",
+	}
-	};
-	options[0].optionString = construct_boot_classpath("-Xbootclasspath:", boot_cp_arr, ARRAY_SIZE(boot_cp_arr));
 
-	// micorg is purposefully after the apk, so that we get the correct resources.arsc
+	// microg is purposefully after the apk, so that we get the correct resources.arsc
-	// TODO: request resources.arsc from concrete classloader instead of taking the first one in classpath
+	// TODO: request resources.arsc from concrete apk instead of taking the first one in classpath
-	char *cp_array[] = {
+	options[1].optionString = construct_classpath("-Djava.class.path=", (char *[]){api_impl_jar, apk_classpath, microg_apk}, 3);
-		api_impl_jar,
-		apk_classpath,
-		microg_apk,
-	};
-	options[1].optionString = construct_classpath("-Djava.class.path=", "", cp_array, ARRAY_SIZE(cp_array));
 	options[2].optionString = "-verbose:jni";
 
 	args.options = options;
@@ -111,14 +87,17 @@ JNIEnv* create_vm(char *api_impl_jar, char *apk_classpath, char *microg_apk) {
 	return env;
 }
 
-// this is an extern "C" function which we added to art in order to simplify calling the underlying c++ method,
+// this is exported by the shim bionic linker
-// which we use to inject our JNI library into the correct classloader without having to load it from java code
+void dl_parse_library_path(const char *path, char *delim);
-bool HAXX__JavaVMExt__LoadNativeLibrary(JNIEnv* env, char *path, jobject class_loader, char** error_msg);
+
+#define DOT_LOCAL_DATA_PATH "/.local/share/android_translation_layer"
+#define REL_DEX_INSTALL_PATH "/../java/dex"
+
+#define REL_API_IMPL_JAR_INSTALL_PATH "/android_translation_layer/api-impl.jar"
+#define REL_API_IMPL_NATIVES_INSTALL_PATH "/android_translation_layer/natives"
+#define REL_MICROG_APK_INSTALL_PATH "/microg/com.google.android.gms.apk"
 
-#define API_IMPL_JAR_PATH_LIBDIR INSTALL_DEXDIR "/translation-layer/api-impl.jar"
 #define API_IMPL_JAR_PATH_LOCAL "./api-impl.jar"
-
-#define MICROG_APK_PATH_LIBDIR INSTALL_DEXDIR "/microg/com.google.android.gms.apk"
 #define MICROG_APK_PATH_LOCAL "./com.google.android.gms.apk"
 
 struct jni_callback_data { char *apk_main_activity_class; uint32_t window_width; uint32_t window_height;};
@@ -131,6 +110,7 @@ static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* h
 		printf(">- [%s]\n", g_file_get_path(files[i]));
 	}
 */
+	char *dex_install_dir;
 	char *api_impl_jar;
 	char *microg_apk;
 	int errno_libdir;
@@ -138,28 +118,91 @@ static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* h
 	int ret;
 
 	char *apk_classpath =  g_file_get_path(files[0]);
+	char *apk_name = g_file_get_basename(files[0]);
 
 	if(apk_classpath == NULL) {
 		printf("error: the specified file path doesn't seem to be valid\n");
 		exit(1);
 	}
 
+	Dl_info libart_so_dl_info;
+	// JNI_CreateJavaVM chosen arbitrarily, what matters is that it's a symbol exported by by libart.so
+	// TODO: we shouldn't necessarily count on art being installed in the same prefix as we are
+	dladdr(JNI_CreateJavaVM, &libart_so_dl_info);
+	// make sure we didn't get NULL
+	if(libart_so_dl_info.dli_fname) {
+		// it's simpler if we can modify the string, so strdup it
+		char *libart_so_full_path = strdup(libart_so_dl_info.dli_fname);
+		*strrchr(libart_so_full_path, '/') = '\0'; // now we should have something like /usr/lib64/art
+		dex_install_dir = malloc(strlen(libart_so_full_path) + strlen(REL_DEX_INSTALL_PATH) + 1); // +1 for NULL
+		strcpy(dex_install_dir, libart_so_full_path);
+		strcat(dex_install_dir, REL_DEX_INSTALL_PATH);
+		free(libart_so_full_path);
+	} else {
+		dex_install_dir = "DIDN'T_GET_SO_PATH_WITH_dladdr_SUS"; // in case we print this as part of some other error, it should be clear what the real cause is
+	}
+
+	char* app_data_dir_base = getenv("ANDROID_APP_DATA_DIR");
+	if(!app_data_dir_base) {
+		const char* home_dir = getenv("HOME");
+		if(home_dir) {
+			app_data_dir_base = malloc(strlen(home_dir) + strlen(DOT_LOCAL_DATA_PATH) + 1); // +1 for NULL
+			strcpy(app_data_dir_base, home_dir);
+			strcat(app_data_dir_base, DOT_LOCAL_DATA_PATH);
+			ret = mkdir(app_data_dir_base, DEFFILEMODE | S_IXUSR | S_IXGRP | S_IXOTH);
+			if (ret) {
+				if(errno != EEXIST) {
+					fprintf(stderr, "error: ANDROID_APP_DATA_DIR not set, and the default directory (%s) coudldn't be created (error: %s)\n", app_data_dir_base, strerror(errno));
+					exit(1);
+				}
+			}
+		} else {
+			fprintf(stderr, "error: ANDROID_APP_DATA_DIR not set, and HOME is not set either so we can't construct the default path\n");
+			exit(1);
+		}
+	}
+	app_data_dir = malloc(strlen(app_data_dir_base) + 1 + strlen(apk_name) + 1 + 1); // +1 for middle '/', +1 for end '/', and +1 for NULL
+	strcpy(app_data_dir, app_data_dir_base);
+	strcat(app_data_dir, "/");
+	// TODO: we should possibly use the app id instead, but we don't currently have a way to get that
+	// arguably both the app id and the apk name might have an issue with duplicates, but if two apks use the same app id, chances are it's less of an issue than when two apks have the same name
+	strcat(app_data_dir, apk_name);
+	strcat(app_data_dir, "/");
+
 	struct stat dont_care;
+	ret = stat(app_data_dir, &dont_care);
+	if(ret) {
+		printf("can't stat %s (%s); we don't currently support parsing certain resources from the apk file, "
+		       "so you will need to create this directory and put in those resources\n"
+		       "some of these resources need to be preprocessed, apktool is mostly fine except it replaces "
+		       "the numeric values in the layout xml files with what it assumes was in the source xml; "
+		       "android studio's apk anylyzer (for example) leaves the 'compiled' integer values that our code expects\n"
+		       "NOTE: if the app you're trying to run doesn't use any such resources, simply create the directory "
+		       "and leave it empty (this will be done automatically once the maunal preprocessing step is not required)\n",
+		       app_data_dir, strerror(errno));
+		exit(1);
+	}
+
 	// check for api-impl.jar and com.google.android.gms.apk in './' first (for running from builddir), and in system install path second
 	ret = stat(API_IMPL_JAR_PATH_LOCAL, &dont_care);
 	errno_localdir = errno;
 	if(!ret) {
-		api_impl_jar = API_IMPL_JAR_PATH_LOCAL;
+		api_impl_jar = strdup(API_IMPL_JAR_PATH_LOCAL); // for running out of builddir; using strdup so we can always safely call free on this
 	} else {
-		ret = stat(API_IMPL_JAR_PATH_LIBDIR, &dont_care); // for running out of builddir
+		char *api_impl_install_dir = malloc(strlen(dex_install_dir) + strlen(REL_API_IMPL_JAR_INSTALL_PATH) + 1); // +1 for NULL
+		strcpy(api_impl_install_dir, dex_install_dir);
+		strcat(api_impl_install_dir, REL_API_IMPL_JAR_INSTALL_PATH);
+
+		ret = stat(api_impl_install_dir, &dont_care);
 		errno_libdir = errno;
 		if(!ret) {
-			api_impl_jar = API_IMPL_JAR_PATH_LIBDIR;
+			api_impl_jar = api_impl_install_dir;
 		} else {
 			printf("error: can't stat api-impl.jar; tried:\n"
 				   "\t\"" API_IMPL_JAR_PATH_LOCAL "\", got - %s\n"
-				   "\t\"" API_IMPL_JAR_PATH_LIBDIR "\", got - %s\n",
+				   "\t\"%s\", got - %s\n",
-			       strerror(errno_localdir), strerror(errno_libdir));
+			       strerror(errno_localdir), 
+			       api_impl_install_dir, strerror(errno_libdir));
 			exit(1);
 		}
 	}
@@ -167,31 +210,39 @@ static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* h
 	ret = stat(MICROG_APK_PATH_LOCAL, &dont_care);
 	errno_localdir = errno;
 	if(!ret) {
-		microg_apk = MICROG_APK_PATH_LOCAL;
+		microg_apk = strdup(MICROG_APK_PATH_LOCAL); // for running out of builddir; using strdup so we can always safely call free on this
 	} else {
-		ret = stat(MICROG_APK_PATH_LIBDIR, &dont_care); // for running out of builddir
+		char *microg_install_dir = malloc(strlen(dex_install_dir) + strlen(REL_MICROG_APK_INSTALL_PATH) + 1); // +1 for NULL
+		strcpy(microg_install_dir, dex_install_dir);
+		strcat(microg_install_dir, REL_MICROG_APK_INSTALL_PATH);
+
+		ret = stat(microg_install_dir, &dont_care);
 		errno_libdir = errno;
 		if(!ret) {
-			microg_apk = MICROG_APK_PATH_LIBDIR;
+			microg_apk = microg_install_dir;
 		} else {
 			printf("warning: can't stat com.google.android.gms.apk; tried:\n"
 				   "\t\"" MICROG_APK_PATH_LOCAL "\", got - %s\n"
-				   "\t\"" MICROG_APK_PATH_LIBDIR "\", got - %s\n",
+				   "\t\"%s\", got - %s\n",
-			       strerror(errno_localdir), strerror(errno_libdir));
+			       strerror(errno_localdir),
+			       microg_install_dir, strerror(errno_libdir));
 		}
 	}
 
-	//FIXME
+	char *api_impl_natives_dir = malloc(strlen(dex_install_dir) + strlen(REL_API_IMPL_NATIVES_INSTALL_PATH) + 1); // +1 for NULL
-	ret = stat("./data", &dont_care);
+	strcpy(api_impl_natives_dir, dex_install_dir);
-	if(ret) {
+	strcat(api_impl_natives_dir, REL_API_IMPL_NATIVES_INSTALL_PATH);
-		printf("FIXME: the app's data dir is currently hardcoded to \"./data\" (this should arguably be fixed); can't stat \"./data\": %s\n", strerror(errno));
-		printf("also FIXME: \"./data/lib/\" must exist, or we crash\n");
-		printf("please note that the app's data dir currently needs special attention - it's where we read assets and apktool-preprocessed resources from\n");
-		printf("do also note that any app you run will use './data' directly as it's data dir, not './data/<package_name>' - this means you can't run multiple apps at once, unless the contents of their data folders happen to not conflict\n");
-		exit(1);
-	}
 
-	JNIEnv* env = create_vm(api_impl_jar, apk_classpath, microg_apk);
+	char *app_lib_dir = malloc(strlen(app_data_dir) + strlen("/lib") + 1); // +1 for NULL
+	strcpy(app_lib_dir, app_data_dir);
+	strcat(app_lib_dir, "/lib");
+
+	// calling directly into the shim bionic linker to whitelist the app's lib dir as containing bionic-linked libraries
+	dl_parse_library_path(app_lib_dir, ":");
+
+	JNIEnv* env = create_vm(api_impl_jar, apk_classpath, microg_apk, api_impl_natives_dir, app_lib_dir);
+
+	free(app_lib_dir);
 
 	if(!d->apk_main_activity_class) {
 		printf("error: missing required option --launch-activity <activity>.\nyou can specify --help to see the list of options\n");
@@ -227,13 +278,14 @@ static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* h
 	jmethodID getClassLoader = _METHOD((*env)->FindClass(env, "java/lang/Class"), "getClassLoader", "()Ljava/lang/ClassLoader;");
 	jobject class_loader = (*env)->CallObjectMethod(env, handle_cache.view.class, getClassLoader);
 
-	char* reason = NULL;
+	jclass java_runtime_class = (*env)->FindClass(env, "java/lang/Runtime");
-	if (!HAXX__JavaVMExt__LoadNativeLibrary(env, "libtranslation_layer_main.so", _REF(class_loader), &reason)) {
-		printf("[main] dvmLoadNativeCode failed for libtranslation_layer_main.so: %s", reason);
-		exit(1);
-	}
 
-	free(reason);
+	jmethodID getRuntime = _STATIC_METHOD(java_runtime_class, "getRuntime", "()Ljava/lang/Runtime;");
+	jobject java_runtime = (*env)->CallStaticObjectMethod(env, java_runtime_class, getRuntime);
+
+	/* this method is private, but it seems we get away with calling it from C */
+	jmethodID loadLibrary_with_classloader = _METHOD(java_runtime_class, "loadLibrary", "(Ljava/lang/String;Ljava/lang/ClassLoader;)V");
+	(*env)->CallVoidMethod(env, java_runtime, loadLibrary_with_classloader, _JSTRING("translation_layer_main"), class_loader);
 
 	/* -- run the main activity's onCreate -- */