move entry point to C, bypass dalvik executable by directly linking against libdvm.so

.gitignore

@@ -16,3 +16,5 @@ data/res/*
 libnative/*
 *.class
 *.dex
+
+/main

README.md

@@ -12,6 +12,8 @@ note: some apps don't like runtime changes to resolution, and currently GLSurfac
 example with custom width/height: `./launch_activity.sh test_apks/org.happysanta.gd_29.apk 'org/happysanta/gd/GDActivity -w 540 -h 960'`
 
 NOTE: you might need to copy some files out from the apk under `data/`, e.g the `assets` folder
+NOTE: on X11, Gtk might decide to use GLX, which completely messes up our EGL-dependent code.
+If you have a debug build of Gtk, you can use GDK_DEBUG=gl-egl to force the use of EGL
 
 when it doesn't work:  
 if you are trying to launch a random app, chances are that we are missing implementations for some  

doc/Architecture.md

@@ -1,26 +1,25 @@
 #### directory structure
 
-`dalvik/*dalvik` - helper scripts for running bytecode in the dalvik VM
+`dalvik/*dalvik` - helper scripts for running bytecode in the dalvik VM (not used anymore, we link against `libdvm.so` directly now)
-`dalvik/linux-${arch}` - pre-complied dalvik from https://gitlab.com/Mis012/dalvik_standalone
+`dalvik/linux-${arch}/` - pre-complied dalvik from https://gitlab.com/Mis012/dalvik_standalone
 
 `arsc_parser/` - Java .arsc parser I found somewhere, with fixes (should eventually get replaced by C code)
-`data` - the equivalent of `/data/data/${app-package-name}/`; TODO - use `data/${app-package-name}` instead 
+`data/` - the equivalent of `/data/data/${app-package-name}/`; TODO - use `data/${app-package-name}` instead 
 to allow for storing the data of multiple apps at the same time
 `data/lib/` - hardcoded location, libraries under which are assumed to be linked against bionic 
 (and will therefore be loaded with a shim bionic linker)
-`doc` - documentation
+`doc/` - documentation
-`jars` - when we want to link against dalvik core java libs, we need to keep their non-dex versions here
+`jars/` - when we want to link against dalvik core java libs, we need to keep their non-dex versions here
-`jni` - C code implementing things which it doesn't make sense to do in Java (ideally this would be most things)
+`jni/` - C code implementing things which it doesn't make sense to do in Java (ideally this would be most things)
-`libandroid-src` - C code implementing `libandroid.so` (this is needed by most JNI libs which come with android apps)
+`libandroid-src/` - C code implementing `libandroid.so` (this is needed by most JNI libs which come with android apps)
-`libnative` - compilation output for `.so` libraries
+`libnative/` - compilation output for our `.so` libraries (currently `libtranslation_layer_main.so` and `libandroid.so`)
-`main-src/org/launch/main.java` - the entry point; TODO - at this point it just calls into the C entry point, 
+`main-src/` - Java code implementing the android APIs
-so should probably move the entry point to C
+`src/` - code for the main executable, which sets stuff up (including the JVM) and launches the activity specified on the cmdline
-`main-src/` (rest) - Java code implementing the android APIs
+`test-apks/` - all apks known to somewhat work, except paid proprietary ones which we sadly can't really include
-`test-apks` - all apks known to somewhat work, except paid proprietary ones
 
-`./com.google.android.gms.apk`: microg; stopgap solution to run apps with GSF dependencies
+`./com.google.android.gms.apk`: microg; stopgap solution to run apps with GSF dependencies (fwiw, the actual solution will likely also use MicroG)
-`./*.dex` - compiled Java code; `hax_arsc_parser.dex` corresponds to `hax_arsc_parser.dex/`, `main.dex` 
+`./*.dex` - compiled Java code; `hax_arsc_parser.dex` corresponds to `hax_arsc_parser.dex/`,
-corresponds to `main-src/org/launch/main.java` and `hax.dex` corresponds to the rest of `main-src`
+`hax.dex` corresponds to `main-src`
 
 #### philosophy
 
@@ -42,50 +41,32 @@ against)
 #### current control flow (to be refined)
 
 1. the user executes `launch_activity.sh`  
-(contents:  
+this wrapper accomplishes the following things:
-```sh
-LD_PRELOAD=libpthread_bio.so:libbsd.so.0 ./dalvik/dalvik -verbose:jni -cp hax_arsc_parser.dex:hax.dex:main.dex:${1}:com.google.android.gms.apk org/launch/main -l ${2}
-```
-)  
-this line accomplishes the following things:
 	- `LD_PRELOAD` makes sure that `libbsd` (which provides some functions that bionic would on android) is loaded, even though bionic-linked libraries don't know they want it
 	- `LD_PRELOAD` also loads libpthread_bio.so, which is a shim for translating libpthread calls made by bionic-linked libraries to calls to glibc/musl libpthread
-	- `./dalvik/dalvik` is a helper script for launching the dalvik vm with the right `LD_LIBRARY_PATH`, `bootclasspath`, etc
+	- `ANDROID_ROOT` and `ANDROID_DATA` get set, both of which are required by dalvik
+	- `BIONIC_LD_LIBRARY_PATH` gets set to the path to the android app's lib folder (currently any app's data directory is `data/`) so that the shim bionic linker knows which .so files must not be loaded by the system linker
+	- `LD_LIBRARY_PATH` is set up such that both the host linker and the shim bionic linker can find any non-system libraries they will need
+	- `./main ${1} -l ${2}` calls the main executable and instructs it to load the apk file `${1}` into classpath before launching the activity `${2}` using the JVM
+
+2. the executable is compiled from `src/main.c`:
+	1. `int main(int argc, char **argv)` sets up a GtkApllication, cmdline handling, and calls `g_application_run`
+	2. GtkApplication glue parses the cmdline and calls `static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* hint, struct jni_callback_data *d)`
+	3. `static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* hint, struct jni_callback_data *d)`:
+		1. constructs the classpath from the following:
 			- `hax_arsc_parser.dex` is dalvik bytecode implementing .arsc parsing duties (to be replaced by C code eventually)
 			- `hax.dex` contains all the implementations of android framework functions
-	- `main.dex` contains the Java entry point `public static void main(String[])`
+			- `%s` will be substituted by the path to the app's apk (passed to us on cmdline), making the bytecode within (and resources.arsc, which is currently the only other file read straight from the apk) available in classpath
-	- `${1}` will be substituted by the path to the app's apk, making the bytecode within (and resources.arsc, which is currently the only other file read straight from the apk) available in classpath
+			- `com.google.android.gms.apk` is the path to a microG apk, needed for apps with a dependency on GSF; this is specified after the app's apk so that the the app's apk is the first zip file in the classpath (needed for getting the right resources.arsc, TODO: ask for the classloader which loaded the activity that was specified on the cmdline)
-	- `com.google.android.gms.apk` is the path to a microG apk, needed for apps with a dependency on GSF; this is specified after the app's apk so that the the app's apk is the first zip file in the classpath (needed for getting the right resources.arsc)
+		2. contructs other options for and launches the dalvik virtual machine
-	- `org/launch/main` is the class which contains the Java entry point, this tells dalvik to look for it there
+		3. load `libtrasnlation_layer_main.so` using the internal function `dvmLoadNativeCode` (this contains our native methods, and dalvik will only search in libraries that were loaded using this or System.load) (NOTE: care is taken to register this with the right classloader so the java code which declares the native methods is the java code that can use them)
-	- `-l ${2}` is cmdline argument list to be passed to the entry point function, which passes it to the C entry point
+		4. sets up a JNI handle cache
+		5. sets display size to be passed to apps according to optionally specified window size (some apps handle runtime resizing poorly, and our EGL code currently doesn't handle it at all)
+		6. creates a Gtk Window with said size and shows it on screen
+		7. calls the OnCreate method of the class specified with the `-l` option
 
-2. the Java entry point is executed by dalvik  
+3. the Activity specfified with `-l` calls various android APIs, which we hopefully implement; 
-(contents:  
+typically, it will set up some android widgets, which will appear in the Gtk window as Gtk widgets
-```Java
-public class main {
-	public static void main(String[] args) {
-		System.load("libnative/org_launch_main.so");
-		real_main(args);
-	}
-
-	public static native void real_main(String[] args);
-}
-M
-```
-);  
-
-3. execution jumps to the C entry point in `jni/org_launch_main.c`
-	1. the function `JNIEXPORT void JNICALL Java_org_launch_main_real_1main(JNIEnv *env, jclass this_class, jobjectArray args)` fixes up argc/argv in preparation for removing the Java entry point in favor of a  C entry point, and calls `static int main(int argc, char **argv, JNIEnv *env)`
-	2. `static int main(int argc, char **argv, JNIEnv *env)` sets up a GtkApllication, cmdline handling, and calls `g_application_run`
-	3. GtkApplication glue parses the cmdline and calls `static void activate(GtkApplication *app, struct jni_callback_data *d)`
-	4. `static void activate(GtkApplication *app, struct jni_callback_data *d)`:
-		1. sets up a JNI handle cache
-		2. sets display size to be passed to apps according to optionally specified window size
-		3. creates a Gtk Window with said size and shows it on screen
-		4. calls the OnCreate method of the class specified with the `-l` option
-
-4. the Activity specfified with `-l` calls various android APIs, which we hopefully implement; 
-typically, it will set up some android widgets, which will appear in the Gtk windows as Gtk widgets
 
 5. you, the user, interact with the Gtk widgets, which the app might have registered callbacks for; 
 as long as our implementation of android APIs is good enough for what the app needs, you can use the app

src/main.c

@@ -0,0 +1,251 @@
+#include <gtk/gtk.h>
+
+#include "../jni/defines.h"
+#include "../jni/util.h"
+
+#include "../jni/org_launch_main.h"
+#include "../jni/android_app_Activity.h"
+
+#include <dlfcn.h>
+
+GtkWidget *window;
+
+// standard Gtk Application stuff, more or less
+
+gboolean app_exit(GtkWindow* self, JNIEnv *env) // TODO: do more cleanup?
+{
+	/* -- run the main activity's onDestroy -- */
+
+	(*env)->CallVoidMethod(env, handle_cache.apk_main_activity.object, handle_cache.apk_main_activity.onDestroy, NULL);
+
+	return false;
+}
+
+// FIXME: used by hacks in GLSurfaceView
+int FIXME__WIDTH;
+int FIXME__HEIGHT;
+
+char *construct_classpath(char *prefix, char *path_to_prepend, char **cp_array, size_t len)
+{
+	size_t result_len = strlen(prefix);
+	for(int i = 0; i < len; i++) {
+		result_len += strlen(path_to_prepend) + strlen(cp_array[i]) + 1; // the 1 is for either : or the final \0
+	}
+
+	char *result = malloc(result_len);
+	strcpy(result, prefix);
+	for(int i = 0; i < len; i++) {
+		strcat(result, path_to_prepend);
+		strcat(result, cp_array[i]);
+		if (i < (len - 1))
+			strcat(result, ":");
+	}
+
+	return result;
+}
+
+char *construct_boot_classpath(char *prefix, char **cp_array, size_t len)
+{
+	char *android_root_path = getenv("ANDROID_ROOT");
+	char *framework_dir = "/framework/";
+	char *framework_dir_path = malloc(strlen(android_root_path) + strlen(framework_dir) + 1);
+	strcpy(framework_dir_path, android_root_path);
+	strcat(framework_dir_path, framework_dir);
+
+	char *result = construct_classpath(prefix, framework_dir_path, cp_array, len);
+	free(framework_dir_path);
+
+	return result;
+}
+
+JNIEnv* create_vm(char *apk_classpath) {
+	JavaVM* jvm;
+	JNIEnv* env;
+	JavaVMInitArgs args;
+	JavaVMOption options[3];
+	args.version = JNI_VERSION_1_6;
+	args.nOptions = 3;
+
+	// TODO: should probably not hardcode this
+	char *boot_cp_arr[] = {
+		"core-hostdex.jar",
+		"core-junit-hostdex.jar",
+		"conscrypt-hostdex.jar",
+		"okhttp-hostdex.jar",
+		"bouncycastle-hostdex.jar",
+		"apachehttp-hostdex.jar",
+		"apache-xml-hostdex.jar",
+	};
+	options[0].optionString = construct_boot_classpath("-Xbootclasspath:", boot_cp_arr, ARRAY_SIZE(boot_cp_arr));
+
+	// or this
+	char *cp_array[] = {
+		".",
+		"hax_arsc_parser.dex",
+		"hax.dex",
+		"%s",
+		"com.google.android.gms.apk",
+	};
+	char *cp_format_str = construct_classpath("-Djava.class.path=", "", cp_array, ARRAY_SIZE(cp_array));
+	size_t cp_len = strlen(cp_format_str) - strlen("%s") + strlen(apk_classpath) + 1;
+	options[1].optionString = malloc(cp_len);
+	snprintf(options[1].optionString, cp_len, cp_format_str, apk_classpath);
+
+	options[2].optionString = "-verbose:jni";
+
+	args.options = options;
+	args.ignoreUnrecognized = JNI_FALSE;
+
+	int ret = JNI_CreateJavaVM(&jvm, (void **)&env, &args);
+	if (ret<0){
+		printf("Unable to Launch JVM\n");
+    } else {
+		printf("JVM launched successfully\n");
+	}
+	return env;
+}
+
+bool _Z17dvmLoadNativeCodePKcP6ObjectPPc(const char* pathName, void* classLoader, char** detail);
+void * _Z20dvmDecodeIndirectRefP6ThreadP8_jobject(void* self, jobject jobj);
+void * _Z13dvmThreadSelfv(void);
+
+struct jni_callback_data { char *apk_main_activity_class; uint32_t window_width; uint32_t window_height;};
+static void open(GtkApplication *app, GFile** files, gint nfiles, const gchar* hint, struct jni_callback_data *d)
+{
+//TODO: pass all files to classpath
+/*
+	printf("nfiles: %d\n", nfiles);
+	for(int i = 0; i < nfiles; i++) {
+		printf(">- [%s]\n", g_file_get_path(files[i]));
+	}
+*/
+	char *apk_classpath =  g_file_get_path(files[0]);
+
+	if(apk_classpath == NULL) {
+		printf("error: the specified file path doesn't seem to be valid\n");
+		exit(1);
+	}
+
+	JNIEnv* env = create_vm(apk_classpath);
+
+	if(!d->apk_main_activity_class) {
+		printf("error: missing required option --launch-activity <activity>.\nyou can specify --help to see the list of options\n");
+		exit(1);
+	}
+
+	set_up_handle_cache(env, d->apk_main_activity_class);
+
+	jclass display_class = (*env)->FindClass(env, "android/view/Display");
+	_SET_STATIC_INT_FIELD(display_class, "window_width", d->window_width);
+	_SET_STATIC_INT_FIELD(display_class, "window_height", d->window_height);
+
+	FIXME__WIDTH = d->window_width;
+	FIXME__HEIGHT = d->window_height;
+
+	window = gtk_application_window_new (app);
+	(*env)->CallVoidMethod(env, handle_cache.apk_main_activity.object, handle_cache.apk_main_activity.set_window, _INTPTR(window));
+	if((*env)->ExceptionCheck(env))
+		(*env)->ExceptionDescribe(env);
+
+	gtk_window_set_title(GTK_WINDOW(window), "com.example.demo_application");
+	gtk_window_set_default_size(GTK_WINDOW(window), d->window_width, d->window_height);
+	g_signal_connect(window, "close-request", G_CALLBACK (app_exit), env);
+
+//	TODO: set icon according to how android gets it for the purposes of displaying it in the launcher
+//	gtk_window_set_icon_name(window, "weather-clear");
+
+	gtk_widget_show(window);
+
+	/* -- register our JNI library under the appropriate classloader -- */
+
+	/* 'android/view/View' is part of the "hax.dex" package, any other function from that package would serve just as well */
+	jmethodID getClassLoader = _METHOD((*env)->FindClass(env, "java/lang/Class"), "getClassLoader", "()Ljava/lang/ClassLoader;");
+	jobject class_loader = (*env)->CallObjectMethod(env, handle_cache.view.class, getClassLoader);
+
+	char* reason = NULL;
+	if (!_Z17dvmLoadNativeCodePKcP6ObjectPPc("libtranslation_layer_main.so", _Z20dvmDecodeIndirectRefP6ThreadP8_jobject(_Z13dvmThreadSelfv(), class_loader), &reason)) {
+		printf("[main] dvmLoadNativeCode failed for libtranslation_layer_main.so: %s", reason);
+		exit(1);
+	}
+
+	/* -- run the main activity's onCreate -- */
+
+	(*env)->CallVoidMethod(env, handle_cache.apk_main_activity.object, handle_cache.apk_main_activity.onCreate, NULL);
+	if((*env)->ExceptionCheck(env))
+		(*env)->ExceptionDescribe(env);
+
+//	TODO: some apps wait for this to actually do stuff
+/*	(*env)->CallVoidMethod(env, handle_cache.apk_main_activity.object, handle_cache.apk_main_activity.onWindowFocusChanged, true);
+	if((*env)->ExceptionCheck(env))
+		(*env)->ExceptionDescribe(env);*/
+}
+
+static void activate(GtkApplication *app, struct jni_callback_data *d)
+{
+	printf("error: usage: ./main [app.apk] [path/to/activity]\nyou can specify --help to see the list of options\n");
+	exit(1);
+}
+
+void init_cmd_parameters(GApplication *app, struct jni_callback_data *d)
+{
+  const GOptionEntry cmd_params[] =
+  {
+    {
+      .long_name = "launch-activity",
+      .short_name = 'l',
+      .flags = G_OPTION_FLAG_NONE,
+      .arg = G_OPTION_ARG_STRING,
+      .arg_data = &d->apk_main_activity_class,
+      .description = "the fully quilifed name of the activity you wish to launch (usually the apk's main activity)",
+      .arg_description = NULL,
+    },
+    {
+      .long_name = "window-width",
+      .short_name = 'w',
+      .flags = G_OPTION_FLAG_NONE,
+      .arg = G_OPTION_ARG_INT,
+      .arg_data = &d->window_width,
+      .description = "window width to launch with (some apps react poorly to runtime window size adjustments)",
+      .arg_description = NULL,
+    },
+    {
+      .long_name = "window-height",
+      .short_name = 'h',
+      .flags = G_OPTION_FLAG_NONE,
+      .arg = G_OPTION_ARG_INT,
+      .arg_data = &d->window_height,
+      .description = "window height to launch with (some apps react poorly to runtime window size adjustments)",
+      .arg_description = NULL,
+    },
+    {NULL}
+  };
+
+  g_application_add_main_option_entries (G_APPLICATION (app), cmd_params);
+}
+
+int main(int argc, char **argv/*, JNIEnv *env*/)
+{
+ 	GtkApplication *app;
+ 	int status;
+
+	struct jni_callback_data *callback_data = malloc(sizeof(struct jni_callback_data));
+	callback_data->apk_main_activity_class = NULL;
+	callback_data->window_width = 960;
+	callback_data->window_height = 540;
+
+ 	app = gtk_application_new("com.example.demo_application", G_APPLICATION_NON_UNIQUE | G_APPLICATION_HANDLES_OPEN);
+
+	// cmdline related setup
+	init_cmd_parameters(G_APPLICATION(app), callback_data);
+	g_application_set_option_context_summary(G_APPLICATION(app), "actual usage:\nLD_PRELOAD=libpthread_bio.so ./dalvik/dalvik -verbose:jni -cp hax_arsc_parser.dex:hax_xmlpull.dex:hax.dex:main.dex:${1}:com.google.android.gms.apk org/launch/main ${2}\nwhere ${1} is the path to the apk and ${2} is the cmdline");
+
+ 	g_signal_connect(app, "activate", G_CALLBACK (activate), callback_data);
+ 	g_signal_connect(app, "open", G_CALLBACK (open), callback_data);
+ 	status = g_application_run(G_APPLICATION(app), argc, argv);
+ 	g_object_unref(app);
+
+ 	return status;
+}
+
+const char dl_loader[] __attribute__((section(".interp"))) =
+    "/lib/ld-linux.so.2";