Files
UnrealEngineUWP/Engine/Source/Programs/AutomationTool/Android/AndroidPlatform.Automation.cs
Ben Marsh 6748a24fb1 Copying //UE4/Dev-Build to //UE4/Dev-Main (Source: //UE4/Dev-Build @ 3232619)
#lockdown Nick.Penwarden
#rb none

==========================
MAJOR FEATURES + CHANGES
==========================

Change 3121996 on 2016/09/12 by Ben.Marsh

	Add support for Visual Studio 2017 (aka "15"; assuming consistent naming with other versions until final name is announced).

	* Compiler, STL implementation and CRT are binary compatible with VS2015 (see https://blogs.msdn.microsoft.com/vcblog/2016/08/24/c1417-features-and-stl-fixes-in-vs-15-preview-4/), so no new third-party libraries needed so far. WindowsPlatform.GetVisualStudioCompilerVersionName() returns "2015" as a result.
	* Default compiler for compiling and generating project files is still VS 2015 for now. Pass -2017 on the command line to GenerateProjectFiles.bat to generate VS2017 projects. Projects generated for VS2017 will use the 2017 compiler by default.
	* Visual Studio source code accessor can talk to VS 2017 instances.
	* Added a VS2017 configuration for UnrealVS, and added precompiled vsix package.
	* Switched GetVSComnTools to check the SOFTWARE\Microsoft\VisualStudio\SxS\VS7 registry key rather than the individual product install registry key. "15" doesn't seem to have it's own "InstallDir" key, but this system seems to work for all versions of Visual Studio (including previous releases of VS Express).
	* Removed ATL dependency from VisualStudioSourceCodeAccessor. It's not installed with VS by default any more, and is only used for a couple of smart pointer classes.

	Tested running the editor and packaging TP_Flying for Win64. Packaging from the editor still defaults to using the 2015 compiler, so ConfigureToolchain() needs to be overriden from the .target.cs file if multiple Visual Studio versions are installed.

Change 3189363 on 2016/11/07 by Ben.Marsh

	Consolidate functionality for determining the path to MSBuild.exe to use for compiling UE4 tools into a single batch file (GetMSBuildToolPath) and fix "Clean" not working on PS4 due to include/library paths being set to something by the Visual Studio environment.

Change 3210598 on 2016/11/27 by Ben.Marsh

	UBT: Prevent the name of each file compiled being output twice on XboxOne. Compiler already outputs this string; the action doesn't need to.

Change 3210601 on 2016/11/27 by Ben.Marsh

	PR #2967: Add silent version of switch game version (Contributed by EricLeeFriedman)

Change 3210602 on 2016/11/27 by Ben.Marsh

	PR #2964: GitDependencies shouldn't try to clean up working directory files that are excluded or ignored (Contributed by joelmcginnis)

Change 3210605 on 2016/11/27 by Ben.Marsh

	UGS: Add a warning when syncing latest would remove changes that have been authored locally. Typically happens when working with precompiled binaries.

Change 3211656 on 2016/11/28 by Ben.Marsh

	UBT: Move ModuleRules and TargetRules into their own file.

Change 3211797 on 2016/11/28 by Ben.Marsh

	UBT: Remove utility functions from TargetRules for checking different classes of target types. Moving TargetRules to be data-only.

Change 3211833 on 2016/11/28 by Ben.Marsh

	UBT: Remove overridable configuration name from target rules. This feature is not used anywhere.

Change 3211859 on 2016/11/28 by Ben.Marsh

	UBT: Deprecate the GetGeneratedCodeVersion() callback in favor of a member variable instead.

Change 3211942 on 2016/11/28 by Ben.Marsh

	UBT: Remove legacy code which tries to change the output paths for console binaries. Output paths for monolithic binaries are always in the project folder now.

Change 3215333 on 2016/11/30 by Ben.Marsh

	UBT: Replace the GetSupportedPlatforms() callback on TargetRules with a SupportedPlatforms attribute. Since a TargetRules object can only be instantiated with an actual platform, it doesn't make sense for it to be an instance method.

Change 3215482 on 2016/11/30 by Ben.Marsh

	UBT: Remove the GetSupportedConfigurations() callback on the TargetRules class. A configuration is required to construct a TargetRules instance, so it doesn't make sense to need to call an instance method to find out which configurations are supported.

Change 3215743 on 2016/11/30 by Ben.Marsh

	UBT: Deprecate the TargetRules.ShouldCompileMonolithic() function: this function requires access to the global command line to operate correctly, which prevents creating target-specific instances, and does not use the platform/configuration passed into the TargetRules constructor.

	Rather than being a callback, the LinkType field can now be set to TargetLinkType.Modular or TargetLinkType.Monolithic from the constructor as appropriate. The default value (TargetLinkType.Default) results in the default link type for the target type being used. Parsing of the command-line overrides is now done when building the TargetDescriptor.

Change 3215778 on 2016/11/30 by Ben.Marsh

	UBT: Mark overrides of the TargetRules.GetModulesToPrecompile method as obsolete.

Change 3217681 on 2016/12/01 by Ben.Marsh

	UAT: Prevent UE4Build deleting .modules files when running with the -Clean argument; these files are artifacts generated by UBT itself, not by the exported XGE script.

Change 3217723 on 2016/12/01 by Ben.Marsh

	UBT: Run pre- and post-build steps for all plugins that are being built, not just those that are enabled.

Change 3217930 on 2016/12/01 by Ben.Marsh

	UGS: Add a perforce settings window, allowing users to set optional values for tuning Perforce performance on unreliable connections.

Change 3218762 on 2016/12/02 by Ben.Marsh

	Enable warnings whenever an undefined macro is used in a constant expression inside an #if or #elif directive, and fix existing violations.

Change 3219161 on 2016/12/02 by Ben.Marsh

	Core: Use the directory containing the current module to derive the UE4 base directory, rather than the executable directory. Allows UE4 to be hosted by a process in a different directory.

Change 3219197 on 2016/12/02 by Ben.Marsh

	Core: When loading a DLL from disk, convert any relative paths to absolute before calling LoadLibrary. The OS resolves these paths relative to the directory containing the process executable -- not the working directory -- so paths need to be absolute to allow UE4 to be hosted by a process elsewhere.

Change 3219209 on 2016/12/02 by Ben.Marsh

	Replace some calls to LoadLibrary() with FPlatformProcess::GetDllHandle(). The UE4 function makes sure that relative paths are resolved relative to the correct base directory, which is important when the host executable is not in Engine/Binaries/Win64.

Change 3219610 on 2016/12/02 by Ben.Marsh

	Add the -q (quiet) option to the Mac unzip command, since it's creating too much log output to be useful.

Change 3219731 on 2016/12/02 by Ben.Marsh

	UBT: Add option to disable IWYU checks regarding the use of monolithic headers (Engine.h, UnrealEd.h, etc...) and including the matching header for a cpp file first. bEnforceIWYU can be set to false in UEBuildConfiguration or on a per-module basis in the module rules.

Change 3220796 on 2016/12/04 by Ben.Marsh

	Remove PrepForUATPackageOrDeploy from the UEBuildDeploy base class. It never has to be accessed through the base class anyway.

Change 3220825 on 2016/12/04 by Ben.Marsh

	UBT: Change all executors to derive from a common base class (ActionExecutor).

Change 3220834 on 2016/12/04 by Ben.Marsh

	UBT: Remove the global CommandLineContains() function.

Change 3222706 on 2016/12/05 by Ben.Marsh

	Merging CL 3221949 from //UE4/Release-4.14: Fixes to code analysis template causing problems with stock install of VS2017.

Change 3222712 on 2016/12/05 by Ben.Marsh

	Merging CL 3222021 from //UE4/Release-4.14: Change detection of MSBuild.exe path to match GetMSBuildPath.bat

Change 3223628 on 2016/12/06 by Ben.Marsh

	Merging CL 3223369 from 4.14 branch: Use the same logic as GetMsBuildPath.bat inside FDesktopPlatformBase to determine path to MSBuild.exe

Change 3223817 on 2016/12/06 by Ben.Marsh

	Remove non-ANSI characters from source files. Compiler/P4 support is patchy for this, and we want to avoid failing prey to different codepages resulting in different interpretations of the source text.

Change 3224046 on 2016/12/06 by Ben.Marsh

	Remove the need for the iOS/TVOS deployment instances to have an IOSPlatformContext instance. The only dependency between the two -- a call to GetRequiredCapabilities() -- is now implemented by querying the INI file for the supported architectures when neeeded.

Change 3224792 on 2016/12/07 by Ben.Marsh

	UBT: Touch PCH wrapper files whenever the file they include is newer rather than writing the timestamp for the included file into it as a comment. Allows use of ccache and similar tools.

Change 3225212 on 2016/12/07 by Ben.Marsh

	UBT: Move settings required for deployment into the UEBuildDeployTarget class, allowing them to be serialized to and from a file the intermediate directory without having to construct a phony UEBuildTarget to deploy.

	Deployment is now performed by a method on UEBuildPlatform, rather than having to create a UEBuildPlatformContext and using that to create a UEBuildDeploy object.

	The -prepfordeploy UBT invocation from UAT, previously done by the per-platform PostBuildTarget() callback when building with XGE, is replaced by running UBT with a path to the serialized UEBuildDeployTarget object, and can be done in a platform agnostic manner.

Change 3226310 on 2016/12/07 by Ben.Marsh

	PR #3015: Fixes wrong VSC++ flags being passed for .c files (Contributed by badlogic)

Change 3228273 on 2016/12/08 by Ben.Marsh

	Update copyright notices for QAGame.

Change 3229166 on 2016/12/09 by Ben.Marsh

	UBT: Rewritten config file parser. No longer requires hard-coded list of sections to be parsed, but parses them on demand. Measured 2x faster read speeds (largely due to eliminating construction of temporary string objects when parsing each line, to trim whitespace and so on). Also includes an attribute-driven parser, which allows reading named config values for marked up fields in an object.

Change 3230601 on 2016/12/12 by Ben.Marsh

	Swarm: Change Swarm AgentInterface to target .NET framework 4.5, to remove dependency on having 4.0 framework installed.

Change 3230737 on 2016/12/12 by Ben.Marsh

	UAT: Stop UE4Build deriving from CommandUtils. Confusing pattern, and causes problems trying to access instance variables that are only set for build commands.

Change 3230751 on 2016/12/12 by Ben.Marsh

	UAT: Move ParseParam*() functions which use the instanced parameter list from CommandUtils to BuildCommand, since that's the only thing that it's instanced for.

Change 3230804 on 2016/12/12 by Ben.Marsh

	UBT: Add the IsPromotedBuild flag to Build.version, and only set the bFormalBuild flag in UBT if it's set. This allows UGS users to avoid having to compile separate RC files for each output binary.

Change 3230831 on 2016/12/12 by Ben.Marsh

	UGS: Warn when trying to switch streams if files are checked out.

Change 3231281 on 2016/12/12 by Chad.Garyet

	Fixing a bug where .modules files were getting put into receipts with their absolute path instead of their relative one

Change 3231496 on 2016/12/12 by Ben.Marsh

	Disable code analysis in CrashReportProcess; causes warnings when compiled with VS2015.

Change 3231979 on 2016/12/12 by Ben.Marsh

	UBT: Suppress LNK4221 when generating import libraries. This can happen often when generating import libraries separately to linking.

Change 3232619 on 2016/12/13 by Ben.Marsh

	Fix "#pragma once in main file" errors on Mac, which are occurring in //UE4/Main.

[CL 3232653 by Ben Marsh in Main branch]
2016-12-13 11:58:16 -05:00

1557 lines
62 KiB
C#

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;
using System.Net.NetworkInformation;
using System.Threading;
using AutomationTool;
using UnrealBuildTool;
using Ionic.Zip;
public class AndroidPlatform : Platform
{
private const int DeployMaxParallelCommands = 6;
private const string TargetAndroidLocation = "obb/";
public AndroidPlatform()
: base(UnrealTargetPlatform.Android)
{
}
private static string GetSONameWithoutArchitecture(ProjectParams Params, string DecoratedExeName)
{
return Path.Combine(Path.GetDirectoryName(Params.ProjectGameExeFilename), DecoratedExeName) + ".so";
}
private static string GetFinalApkName(ProjectParams Params, string DecoratedExeName, bool bRenameUE4Game, string Architecture, string GPUArchitecture)
{
string ProjectDir = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(Params.RawProjectPath.FullName)), "Binaries/Android");
if (Params.Prebuilt)
{
ProjectDir = Path.Combine(Params.BaseStageDirectory, "Android");
}
// Apk's go to project location, not necessarily where the .so is (content only packages need to output to their directory)
string ApkName = Path.Combine(ProjectDir, DecoratedExeName) + Architecture + GPUArchitecture + ".apk";
// if the source binary was UE4Game, handle using it or switching to project name
if (Path.GetFileNameWithoutExtension(Params.ProjectGameExeFilename) == "UE4Game")
{
if (bRenameUE4Game)
{
// replace UE4Game with project name (only replace in the filename part)
ApkName = Path.Combine(Path.GetDirectoryName(ApkName), Path.GetFileName(ApkName).Replace("UE4Game", Params.ShortProjectName));
}
else
{
// if we want to use UE4 directly then use it from the engine directory not project directory
ApkName = ApkName.Replace(ProjectDir, Path.Combine(CmdEnv.LocalRoot, "Engine/Binaries/Android"));
}
}
return ApkName;
}
private static string GetFinalObbName(string ApkName)
{
// calculate the name for the .obb file
string PackageName = GetPackageInfo(ApkName, false);
if (PackageName == null)
{
throw new AutomationException(ExitCode.Error_FailureGettingPackageInfo, "Failed to get package name from " + ApkName);
}
string PackageVersion = GetPackageInfo(ApkName, true);
if (PackageVersion == null || PackageVersion.Length == 0)
{
throw new AutomationException(ExitCode.Error_FailureGettingPackageInfo, "Failed to get package version from " + ApkName);
}
if (PackageVersion.Length > 0)
{
int IntVersion = int.Parse(PackageVersion);
PackageVersion = IntVersion.ToString("0");
}
string ObbName = string.Format("main.{0}.{1}.obb", PackageVersion, PackageName);
// plop the .obb right next to the executable
ObbName = Path.Combine(Path.GetDirectoryName(ApkName), ObbName);
return ObbName;
}
private static string GetDeviceObbName(string ApkName)
{
string ObbName = GetFinalObbName(ApkName);
string PackageName = GetPackageInfo(ApkName, false);
return TargetAndroidLocation + PackageName + "/" + Path.GetFileName(ObbName);
}
private static string GetStorageQueryCommand()
{
if (Utils.IsRunningOnMono)
{
return "shell 'echo $EXTERNAL_STORAGE'";
}
else
{
return "shell \"echo $EXTERNAL_STORAGE\"";
}
}
private static string GetFinalBatchName(string ApkName, ProjectParams Params, string Architecture, string GPUArchitecture, bool bNoOBBInstall, bool bUninstall, UnrealTargetPlatform Target)
{
string Extension = ".bat";
switch (Target)
{
default:
case UnrealTargetPlatform.Win64:
Extension = ".bat";
break;
case UnrealTargetPlatform.Linux:
Extension = ".sh";
break;
case UnrealTargetPlatform.Mac:
Extension = ".command";
break;
}
return Path.Combine(Path.GetDirectoryName(ApkName), (bUninstall ? "Uninstall_" : "Install_") + Params.ShortProjectName + (!bNoOBBInstall ? "_" : "_NoOBBInstall_") + Params.ClientConfigsToBuild[0].ToString() + Architecture + GPUArchitecture + Extension);
}
private List<string> CollectPluginDataPaths(DeploymentContext SC)
{
// collect plugin extra data paths from target receipts
List<string> PluginExtras = new List<string>();
foreach (StageTarget Target in SC.StageTargets)
{
TargetReceipt Receipt = Target.Receipt;
var Results = Receipt.AdditionalProperties.Where(x => x.Name == "AndroidPlugin");
foreach (var Property in Results)
{
// Keep only unique paths
string PluginPath = Property.Value;
if (PluginExtras.FirstOrDefault(x => x == PluginPath) == null)
{
PluginExtras.Add(PluginPath);
Log("AndroidPlugin: {0}", PluginPath);
}
}
}
return PluginExtras;
}
public override void Package(ProjectParams Params, DeploymentContext SC, int WorkingCL)
{
AndroidToolChain ToolChain = new AndroidToolChain(Params.RawProjectPath);
var Architectures = ToolChain.GetAllArchitectures();
var GPUArchitectures = ToolChain.GetAllGPUArchitectures();
bool bMakeSeparateApks = UnrealBuildTool.UEDeployAndroid.ShouldMakeSeparateApks();
var Deploy = new UEDeployAndroid(Params.RawProjectPath);
bool bPackageDataInsideApk = Deploy.PackageDataInsideApk(false);
string BaseApkName = GetFinalApkName(Params, SC.StageExecutables[0], true, "", "");
Log("BaseApkName = {0}", BaseApkName);
// Create main OBB with entire contents of staging dir. This
// includes any PAK files, movie files, etc.
string LocalObbName = SC.StageDirectory.TrimEnd(new char[] {'/', '\\'})+".obb";
// Always delete the target OBB file if it exists
if (File.Exists(LocalObbName))
{
File.Delete(LocalObbName);
}
// Now create the OBB as a ZIP archive.
Log("Creating {0} from {1}", LocalObbName, SC.StageDirectory);
using (ZipFile ObbFile = new ZipFile(LocalObbName))
{
ObbFile.CompressionMethod = CompressionMethod.None;
ObbFile.CompressionLevel = Ionic.Zlib.CompressionLevel.None;
ObbFile.UseZip64WhenSaving = Ionic.Zip.Zip64Option.Never;
int ObbFileCount = 0;
ObbFile.AddProgress +=
delegate(object sender, AddProgressEventArgs e)
{
if (e.EventType == ZipProgressEventType.Adding_AfterAddEntry)
{
ObbFileCount += 1;
Log("[{0}/{1}] Adding {2} to OBB",
ObbFileCount, e.EntriesTotal,
e.CurrentEntry.FileName);
}
};
ObbFile.AddDirectory(SC.StageDirectory+"/"+SC.ShortProjectName, SC.ShortProjectName);
try
{
ObbFile.Save();
}
catch (Exception)
{
Log("Failed to build OBB: " + LocalObbName);
throw new AutomationException(ExitCode.Error_MissingExecutable, "Stage Failed. Could not build OBB {0}. The file may be too big to fit in an OBB (2 GiB limit)", LocalObbName);
}
}
// collect plugin extra data paths from target receipts
Deploy.SetAndroidPluginData(Architectures, CollectPluginDataPaths(SC));
foreach (string Architecture in Architectures)
{
foreach (string GPUArchitecture in GPUArchitectures)
{
string ApkName = GetFinalApkName(Params, SC.StageExecutables[0], true, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "");
if (!SC.IsCodeBasedProject)
{
string UE4SOName = GetFinalApkName(Params, SC.StageExecutables[0], false, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "");
UE4SOName = UE4SOName.Replace(".apk", ".so");
if (FileExists_NoExceptions(UE4SOName) == false)
{
Log("Failed to find game .so " + UE4SOName);
throw new AutomationException(ExitCode.Error_MissingExecutable, "Stage Failed. Could not find .so {0}. You may need to build the UE4 project with your target configuration and platform.", UE4SOName);
}
}
if (!Params.Prebuilt)
{
string CookFlavor = SC.FinalCookPlatform.IndexOf("_") > 0 ? SC.FinalCookPlatform.Substring(SC.FinalCookPlatform.IndexOf("_")) : "";
string SOName = GetSONameWithoutArchitecture(Params, SC.StageExecutables[0]);
Deploy.PrepForUATPackageOrDeploy(Params.RawProjectPath, Params.ShortProjectName, SC.ProjectRoot, SOName, SC.LocalRoot + "/Engine", Params.Distribution, CookFlavor, false);
}
// Create APK specific OBB in case we have a detached OBB.
string DeviceObbName = "";
string ObbName = "";
if (!bPackageDataInsideApk)
{
DeviceObbName = GetDeviceObbName(ApkName);
ObbName = GetFinalObbName(ApkName);
CopyFile(LocalObbName, ObbName);
}
//figure out which platforms we need to create install files for
bool bNeedsPCInstall = false;
bool bNeedsMacInstall = false;
bool bNeedsLinuxInstall = false;
GetPlatformInstallOptions(SC, out bNeedsPCInstall, out bNeedsMacInstall, out bNeedsLinuxInstall);
//helper delegate to prevent code duplication but allow us access to all the local variables we need
var CreateInstallFilesAction = new Action<UnrealTargetPlatform>(Target =>
{
bool bIsPC = (Target == UnrealTargetPlatform.Win64);
// Write install batch file(s).
string BatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", false, false, Target);
string PackageName = GetPackageInfo(ApkName, false);
// make a batch file that can be used to install the .apk and .obb files
string[] BatchLines = GenerateInstallBatchFile(bPackageDataInsideApk, PackageName, ApkName, Params, ObbName, DeviceObbName, false, bIsPC);
File.WriteAllLines(BatchName, BatchLines);
// make a batch file that can be used to uninstall the .apk and .obb files
string UninstallBatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", false, true, Target);
BatchLines = GenerateUninstallBatchFile(bPackageDataInsideApk, PackageName, ApkName, Params, ObbName, DeviceObbName, false, bIsPC);
File.WriteAllLines(UninstallBatchName, BatchLines);
if (Utils.IsRunningOnMono)
{
CommandUtils.FixUnixFilePermissions(BatchName);
CommandUtils.FixUnixFilePermissions(UninstallBatchName);
//if(File.Exists(NoInstallBatchName))
//{
// CommandUtils.FixUnixFilePermissions(NoInstallBatchName);
//}
}
}
);
if (bNeedsPCInstall)
{
CreateInstallFilesAction.Invoke(UnrealTargetPlatform.Win64);
}
if (bNeedsMacInstall)
{
CreateInstallFilesAction.Invoke(UnrealTargetPlatform.Mac);
}
if (bNeedsLinuxInstall)
{
CreateInstallFilesAction.Invoke(UnrealTargetPlatform.Linux);
}
// If we aren't packaging data in the APK then lets write out a bat file to also let us test without the OBB
// on the device.
//String NoInstallBatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", true, false);
// if(!bPackageDataInsideApk)
//{
// BatchLines = GenerateInstallBatchFile(bPackageDataInsideApk, PackageName, ApkName, Params, ObbName, DeviceObbName, true);
// File.WriteAllLines(NoInstallBatchName, BatchLines);
//}
}
}
PrintRunTime();
}
private string[] GenerateInstallBatchFile(bool bPackageDataInsideApk, string PackageName, string ApkName, ProjectParams Params, string ObbName, string DeviceObbName, bool bNoObbInstall, bool bIsPC)
{
string[] BatchLines = null;
string ReadPermissionGrantCommand = "shell pm grant " + PackageName + " android.permission.READ_EXTERNAL_STORAGE";
string WritePermissionGrantCommand = "shell pm grant " + PackageName + " android.permission.WRITE_EXTERNAL_STORAGE";
if (!bIsPC)
{
string OBBInstallCommand = bNoObbInstall ? "shell 'rm -r $EXTERNAL_STORAGE/" + DeviceObbName + "'" : "push " + Path.GetFileName(ObbName) + " $STORAGE/" + DeviceObbName;
Log("Writing shell script for install with {0}", bPackageDataInsideApk ? "data in APK" : "separate obb");
BatchLines = new string[] {
"#!/bin/sh",
"cd \"`dirname \"$0\"`\"",
"ADB=",
"if [ \"$ANDROID_HOME\" != \"\" ]; then ADB=$ANDROID_HOME/platform-tools/adb; else ADB=" +Environment.GetEnvironmentVariable("ANDROID_HOME") + "; fi",
"DEVICE=",
"if [ \"$1\" != \"\" ]; then DEVICE=\"-s $1\"; fi",
"echo",
"echo Uninstalling existing application. Failures here can almost always be ignored.",
"$ADB $DEVICE uninstall " + PackageName,
"echo",
"echo Installing existing application. Failures here indicate a problem with the device \\(connection or storage permissions\\) and are fatal.",
"$ADB $DEVICE install " + Path.GetFileName(ApkName),
"if [ $? -eq 0 ]; then",
"\techo",
bPackageDataInsideApk ? "" : "\techo Grant READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE to the apk for reading OBB file.",
bPackageDataInsideApk ? "" : "\t$ADB $DEVICE " + ReadPermissionGrantCommand,
bPackageDataInsideApk ? "" : "\t$ADB $DEVICE " + WritePermissionGrantCommand,
"\techo",
"\techo Removing old data. Failures here are usually fine - indicating the files were not on the device.",
"\t$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/UE4Game/" + Params.ShortProjectName + "'",
"\t$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/UE4Game/UE4CommandLine.txt" + "'",
"\t$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/" + TargetAndroidLocation + PackageName + "'",
bPackageDataInsideApk ? "" : "\techo",
bPackageDataInsideApk ? "" : "\techo Installing new data. Failures here indicate storage problems \\(missing SD card or bad permissions\\) and are fatal.",
bPackageDataInsideApk ? "" : "\tSTORAGE=$(echo \"`$ADB $DEVICE shell 'echo $EXTERNAL_STORAGE'`\" | cat -v | tr -d '^M')",
bPackageDataInsideApk ? "" : "\t$ADB $DEVICE " + OBBInstallCommand,
bPackageDataInsideApk ? "if [ 1 ]; then" : "\tif [ $? -eq 0 ]; then",
"\t\techo",
"\t\techo Installation successful",
"\t\texit 0",
"\tfi",
"fi",
"echo",
"echo There was an error installing the game or the obb file. Look above for more info.",
"echo",
"echo Things to try:",
"echo Check that the device (and only the device) is listed with \\\"$ADB devices\\\" from a command prompt.",
"echo Make sure all Developer options look normal on the device",
"echo Check that the device has an SD card.",
"exit 1"
};
}
else
{
string OBBInstallCommand = bNoObbInstall ? "shell rm -r %STORAGE%/" + DeviceObbName : "push " + Path.GetFileName(ObbName) + " %STORAGE%/" + DeviceObbName;
Log("Writing bat for install with {0}", bPackageDataInsideApk ? "data in APK" : "separate OBB");
BatchLines = new string[] {
"setlocal",
"set ANDROIDHOME=%ANDROID_HOME%",
"if \"%ANDROIDHOME%\"==\"\" set ANDROIDHOME="+Environment.GetEnvironmentVariable("ANDROID_HOME"),
"set ADB=%ANDROIDHOME%\\platform-tools\\adb.exe",
"set DEVICE=",
"if not \"%1\"==\"\" set DEVICE=-s %1",
"for /f \"delims=\" %%A in ('%ADB% %DEVICE% " + GetStorageQueryCommand() +"') do @set STORAGE=%%A",
"@echo.",
"@echo Uninstalling existing application. Failures here can almost always be ignored.",
"%ADB% %DEVICE% uninstall " + PackageName,
"@echo.",
"@echo Installing existing application. Failures here indicate a problem with the device (connection or storage permissions) and are fatal.",
"%ADB% %DEVICE% install " + Path.GetFileName(ApkName),
"@if \"%ERRORLEVEL%\" NEQ \"0\" goto Error",
"%ADB% %DEVICE% shell rm -r %STORAGE%/UE4Game/" + Params.ShortProjectName,
"%ADB% %DEVICE% shell rm -r %STORAGE%/UE4Game/UE4CommandLine.txt", // we need to delete the commandline in UE4Game or it will mess up loading
"%ADB% %DEVICE% shell rm -r %STORAGE%/" + TargetAndroidLocation + PackageName,
bPackageDataInsideApk ? "" : "@echo.",
bPackageDataInsideApk ? "" : "@echo Installing new data. Failures here indicate storage problems (missing SD card or bad permissions) and are fatal.",
bPackageDataInsideApk ? "" : "%ADB% %DEVICE% " + OBBInstallCommand,
bPackageDataInsideApk ? "" : "if \"%ERRORLEVEL%\" NEQ \"0\" goto Error",
"@echo.",
bPackageDataInsideApk ? "" : "@echo Grant READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE to the apk for reading OBB file.",
bPackageDataInsideApk ? "" : "%ADB% %DEVICE% " + ReadPermissionGrantCommand,
bPackageDataInsideApk ? "" : "%ADB% %DEVICE% " + WritePermissionGrantCommand,
"@echo.",
"@echo Installation successful",
"goto:eof",
":Error",
"@echo.",
"@echo There was an error installing the game or the obb file. Look above for more info.",
"@echo.",
"@echo Things to try:",
"@echo Check that the device (and only the device) is listed with \"%ADB$ devices\" from a command prompt.",
"@echo Make sure all Developer options look normal on the device",
"@echo Check that the device has an SD card.",
"@pause"
};
}
return BatchLines;
}
private string[] GenerateUninstallBatchFile(bool bPackageDataInsideApk, string PackageName, string ApkName, ProjectParams Params, string ObbName, string DeviceObbName, bool bNoObbInstall, bool bIsPC)
{
string[] BatchLines = null;
if (!bIsPC)
{
Log("Writing shell script for uninstall with {0}", bPackageDataInsideApk ? "data in APK" : "separate obb");
BatchLines = new string[] {
"#!/bin/sh",
"cd \"`dirname \"$0\"`\"",
"ADB=",
"if [ \"$ANDROID_HOME\" != \"\" ]; then ADB=$ANDROID_HOME/platform-tools/adb; else ADB=" +Environment.GetEnvironmentVariable("ANDROID_HOME") + "; fi",
"DEVICE=",
"if [ \"$1\" != \"\" ]; then DEVICE=\"-s $1\"; fi",
"echo",
"echo Uninstalling existing application. Failures here can almost always be ignored.",
"$ADB $DEVICE uninstall " + PackageName,
"echo",
"echo Removing old data. Failures here are usually fine - indicating the files were not on the device.",
"$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/UE4Game/" + Params.ShortProjectName + "'",
"$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/UE4Game/UE4CommandLine.txt" + "'",
"$ADB $DEVICE shell 'rm -r $EXTERNAL_STORAGE/" + TargetAndroidLocation + PackageName + "'",
"echo",
"echo Uninstall completed",
"exit 0",
};
}
else
{
Log("Writing bat for uninstall with {0}", bPackageDataInsideApk ? "data in APK" : "separate OBB");
BatchLines = new string[] {
"setlocal",
"set ANDROIDHOME=%ANDROID_HOME%",
"if \"%ANDROIDHOME%\"==\"\" set ANDROIDHOME="+Environment.GetEnvironmentVariable("ANDROID_HOME"),
"set ADB=%ANDROIDHOME%\\platform-tools\\adb.exe",
"set DEVICE=",
"if not \"%1\"==\"\" set DEVICE=-s %1",
"for /f \"delims=\" %%A in ('%ADB% %DEVICE% " + GetStorageQueryCommand() +"') do @set STORAGE=%%A",
"@echo.",
"@echo Uninstalling existing application. Failures here can almost always be ignored.",
"%ADB% %DEVICE% uninstall " + PackageName,
"@echo.",
"echo Removing old data. Failures here are usually fine - indicating the files were not on the device.",
"%ADB% %DEVICE% shell rm -r %STORAGE%/UE4Game/" + Params.ShortProjectName,
"%ADB% %DEVICE% shell rm -r %STORAGE%/UE4Game/UE4CommandLine.txt", // we need to delete the commandline in UE4Game or it will mess up loading
"%ADB% %DEVICE% shell rm -r %STORAGE%/" + TargetAndroidLocation + PackageName,
"@echo.",
"@echo Uninstall completed",
};
}
return BatchLines;
}
public override void GetFilesToArchive(ProjectParams Params, DeploymentContext SC)
{
if (SC.StageTargetConfigurations.Count != 1)
{
throw new AutomationException(ExitCode.Error_OnlyOneTargetConfigurationSupported, "Android is currently only able to package one target configuration at a time, but StageTargetConfigurations contained {0} configurations", SC.StageTargetConfigurations.Count);
}
AndroidToolChain ToolChain = new AndroidToolChain(Params.RawProjectPath);
var Architectures = ToolChain.GetAllArchitectures();
var GPUArchitectures = ToolChain.GetAllGPUArchitectures();
bool bMakeSeparateApks = UnrealBuildTool.UEDeployAndroid.ShouldMakeSeparateApks();
bool bPackageDataInsideApk = new UnrealBuildTool.UEDeployAndroid(Params.RawProjectPath).PackageDataInsideApk(false);
bool bAddedOBB = false;
foreach (string Architecture in Architectures)
{
foreach (string GPUArchitecture in GPUArchitectures)
{
string ApkName = GetFinalApkName(Params, SC.StageExecutables[0], true, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "");
string ObbName = GetFinalObbName(ApkName);
//string NoOBBBatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", true, false);
// verify the files exist
if (!FileExists(ApkName))
{
throw new AutomationException(ExitCode.Error_AppNotFound, "ARCHIVE FAILED - {0} was not found", ApkName);
}
if (!bPackageDataInsideApk && !FileExists(ObbName))
{
throw new AutomationException(ExitCode.Error_ObbNotFound, "ARCHIVE FAILED - {0} was not found", ObbName);
}
SC.ArchiveFiles(Path.GetDirectoryName(ApkName), Path.GetFileName(ApkName));
if (!bPackageDataInsideApk && !bAddedOBB)
{
bAddedOBB = true;
SC.ArchiveFiles(Path.GetDirectoryName(ObbName), Path.GetFileName(ObbName));
}
bool bNeedsPCInstall = false;
bool bNeedsMacInstall = false;
bool bNeedsLinuxInstall = false;
GetPlatformInstallOptions(SC, out bNeedsPCInstall, out bNeedsMacInstall, out bNeedsLinuxInstall);
//helper delegate to prevent code duplication but allow us access to all the local variables we need
var CreateBatchFilesAndArchiveAction = new Action<UnrealTargetPlatform>(Target =>
{
string BatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", false, false, Target);
string UninstallBatchName = GetFinalBatchName(ApkName, Params, bMakeSeparateApks ? Architecture : "", bMakeSeparateApks ? GPUArchitecture : "", false, true, Target);
SC.ArchiveFiles(Path.GetDirectoryName(BatchName), Path.GetFileName(BatchName));
SC.ArchiveFiles(Path.GetDirectoryName(UninstallBatchName), Path.GetFileName(UninstallBatchName));
//SC.ArchiveFiles(Path.GetDirectoryName(NoOBBBatchName), Path.GetFileName(NoOBBBatchName));
}
);
//it's possible we will need both PC and Mac/Linux install files, do both
if (bNeedsPCInstall)
{
CreateBatchFilesAndArchiveAction(UnrealTargetPlatform.Win64);
}
if (bNeedsMacInstall)
{
CreateBatchFilesAndArchiveAction(UnrealTargetPlatform.Mac);
}
if (bNeedsLinuxInstall)
{
CreateBatchFilesAndArchiveAction(UnrealTargetPlatform.Linux);
}
}
}
}
private void GetPlatformInstallOptions(DeploymentContext SC, out bool bNeedsPCInstall, out bool bNeedsMacInstall, out bool bNeedsLinuxInstall)
{
ConfigHierarchy Ini = ConfigCache.ReadHierarchy(ConfigHierarchyType.Engine, DirectoryReference.FromFile(SC.RawProjectPath), SC.StageTargetPlatform.PlatformType);
bool bGenerateAllPlatformInstall = false;
Ini.GetBool("/Script/AndroidRuntimeSettings.AndroidRuntimeSettings", "bCreateAllPlatformsInstall", out bGenerateAllPlatformInstall);
bNeedsPCInstall = bNeedsMacInstall = bNeedsLinuxInstall = false;
if (bGenerateAllPlatformInstall)
{
bNeedsPCInstall = bNeedsMacInstall = bNeedsLinuxInstall = true;
}
else
{
if (HostPlatform.Current.HostEditorPlatform == UnrealTargetPlatform.Mac)
{
bNeedsMacInstall = true;
}
else if (HostPlatform.Current.HostEditorPlatform == UnrealTargetPlatform.Linux)
{
bNeedsLinuxInstall = true;
}
else
{
bNeedsPCInstall = true;
}
}
}
private string GetAdbCommandLine(ProjectParams Params, string SerialNumber, string Args)
{
if (SerialNumber != "")
{
SerialNumber = "-s " + SerialNumber;
}
return string.Format("{0} {1}", SerialNumber, Args);
}
static string LastSpewFilename = "";
public string ADBSpewFilter(string Message)
{
if (Message.StartsWith("[") && Message.Contains("%]"))
{
int LastIndex = Message.IndexOf(":");
LastIndex = LastIndex == -1 ? Message.Length : LastIndex;
if (Message.Length > 7)
{
string Filename = Message.Substring(7, LastIndex - 7);
if (Filename == LastSpewFilename)
{
return null;
}
LastSpewFilename = Filename;
}
return Message;
}
return Message;
}
private IProcessResult RunAdbCommand(ProjectParams Params, string SerialNumber, string Args, string Input = null, ERunOptions Options = ERunOptions.Default)
{
string AdbCommand = Environment.ExpandEnvironmentVariables("%ANDROID_HOME%/platform-tools/adb" + (Utils.IsRunningOnMono ? "" : ".exe"));
if (Options.HasFlag(ERunOptions.AllowSpew) || Options.HasFlag(ERunOptions.SpewIsVerbose))
{
LastSpewFilename = "";
return Run(AdbCommand, GetAdbCommandLine(Params, SerialNumber, Args), Input, Options, SpewFilterCallback: new ProcessResult.SpewFilterCallbackType(ADBSpewFilter));
}
return Run(AdbCommand, GetAdbCommandLine(Params, SerialNumber, Args), Input, Options);
}
private string RunAndLogAdbCommand(ProjectParams Params, string SerialNumber, string Args, out int SuccessCode)
{
string AdbCommand = Environment.ExpandEnvironmentVariables("%ANDROID_HOME%/platform-tools/adb" + (Utils.IsRunningOnMono ? "" : ".exe"));
LastSpewFilename = "";
return RunAndLog(CmdEnv, AdbCommand, GetAdbCommandLine(Params, SerialNumber, Args), out SuccessCode, SpewFilterCallback: new ProcessResult.SpewFilterCallbackType(ADBSpewFilter));
}
public override void GetConnectedDevices(ProjectParams Params, out List<string> Devices)
{
Devices = new List<string>();
IProcessResult Result = RunAdbCommand(Params, "", "devices");
if (Result.Output.Length > 0)
{
string[] LogLines = Result.Output.Split(new char[] { '\n', '\r' });
bool FoundList = false;
for (int i = 0; i < LogLines.Length; ++i)
{
if (FoundList == false)
{
if (LogLines[i].StartsWith("List of devices attached"))
{
FoundList = true;
}
continue;
}
string[] DeviceLine = LogLines[i].Split(new char[] { '\t' });
if (DeviceLine.Length == 2)
{
// the second param should be "device"
// if it's not setup correctly it might be "unattached" or "powered off" or something like that
// warning in that case
if (DeviceLine[1] == "device")
{
Devices.Add("@" + DeviceLine[0]);
}
else
{
CommandUtils.LogWarning("Device attached but in bad state {0}:{1}", DeviceLine[0], DeviceLine[1]);
}
}
}
}
}
/*
private class TimeRegion : System.IDisposable
{
private System.DateTime StartTime { get; set; }
private string Format { get; set; }
private System.Collections.Generic.List<object> FormatArgs { get; set; }
public TimeRegion(string format, params object[] format_args)
{
Format = format;
FormatArgs = new List<object>(format_args);
StartTime = DateTime.UtcNow;
}
public void Dispose()
{
double total_time = (DateTime.UtcNow - StartTime).TotalMilliseconds / 1000.0;
FormatArgs.Insert(0, total_time);
CommandUtils.Log(Format, FormatArgs.ToArray());
}
}
*/
public override bool RetrieveDeployedManifests(ProjectParams Params, DeploymentContext SC, string DeviceName, out List<string> UFSManifests, out List<string> NonUFSManifests)
{
UFSManifests = null;
NonUFSManifests = null;
// Query the storage path from the device
string DeviceStorageQueryCommand = GetStorageQueryCommand();
IProcessResult StorageResult = RunAdbCommand(Params, DeviceName, DeviceStorageQueryCommand, null, ERunOptions.AppMustExist);
String StorageLocation = StorageResult.Output.Trim();
string RemoteDir = StorageLocation + "/UE4Game/" + Params.ShortProjectName;
// Note: appends the device name to make the filename unique; these files will be deleted later during delta manifest generation
// Replace colon with underscore for legal filename (colon may be present for wifi connected devices)
string SanitizedDeviceName = DeviceName.Replace(":", "_");
// Try retrieving the UFS files manifest files from the device
string UFSManifestFileName = CombinePaths(SC.StageDirectory, SC.UFSDeployedManifestFileName + "_" + SanitizedDeviceName);
IProcessResult UFSResult = RunAdbCommand(Params, DeviceName, " pull " + RemoteDir + "/" + SC.UFSDeployedManifestFileName + " \"" + UFSManifestFileName + "\"", null, ERunOptions.AppMustExist);
if (!(UFSResult.Output.Contains("bytes") || UFSResult.Output.Contains("[100%]")))
{
return false;
}
// Try retrieving the non UFS files manifest files from the device
string NonUFSManifestFileName = CombinePaths(SC.StageDirectory, SC.NonUFSDeployedManifestFileName + "_" + SanitizedDeviceName);
IProcessResult NonUFSResult = RunAdbCommand(Params, DeviceName, " pull " + RemoteDir + "/" + SC.NonUFSDeployedManifestFileName + " \"" + NonUFSManifestFileName + "\"", null, ERunOptions.AppMustExist);
if (!(NonUFSResult.Output.Contains("bytes") || NonUFSResult.Output.Contains("[100%]")))
{
// Did not retrieve both so delete one we did retrieve
File.Delete(UFSManifestFileName);
return false;
}
// Return the manifest files
UFSManifests = new List<string>();
UFSManifests.Add(UFSManifestFileName);
NonUFSManifests = new List<string>();
NonUFSManifests.Add(NonUFSManifestFileName);
return true;
}
internal class LongestFirst : IComparer<string>
{
public int Compare(string a, string b)
{
if (a.Length == b.Length) return a.CompareTo(b);
else return b.Length - a.Length;
}
}
public override void Deploy(ProjectParams Params, DeploymentContext SC)
{
foreach (var DeviceName in Params.DeviceNames)
{
string DeviceArchitecture = GetBestDeviceArchitecture(Params, DeviceName);
string GPUArchitecture = GetBestGPUArchitecture(Params, DeviceName);
string ApkName = GetFinalApkName(Params, SC.StageExecutables[0], true, DeviceArchitecture, GPUArchitecture);
// make sure APK is up to date (this is fast if so)
var Deploy = new UEDeployAndroid(Params.RawProjectPath);
if (!Params.Prebuilt)
{
string CookFlavor = SC.FinalCookPlatform.IndexOf("_") > 0 ? SC.FinalCookPlatform.Substring(SC.FinalCookPlatform.IndexOf("_")) : "";
string SOName = GetSONameWithoutArchitecture(Params, SC.StageExecutables[0]);
List<string> Architectures = new List<string>();
Architectures.Add(DeviceArchitecture);
Deploy.SetAndroidPluginData(Architectures, CollectPluginDataPaths(SC));
Deploy.PrepForUATPackageOrDeploy(Params.RawProjectPath, Params.ShortProjectName, SC.ProjectRoot, SOName, SC.LocalRoot + "/Engine", Params.Distribution, CookFlavor, true);
}
// now we can use the apk to get more info
string PackageName = GetPackageInfo(ApkName, false);
// Setup the OBB name and add the storage path (queried from the device) to it
string DeviceStorageQueryCommand = GetStorageQueryCommand();
IProcessResult Result = RunAdbCommand(Params, DeviceName, DeviceStorageQueryCommand, null, ERunOptions.AppMustExist);
String StorageLocation = Result.Output.Trim(); // "/mnt/sdcard";
string DeviceObbName = StorageLocation + "/" + GetDeviceObbName(ApkName);
string RemoteDir = StorageLocation + "/UE4Game/" + Params.ShortProjectName;
// determine if APK out of date
string APKLastUpdateTime = new FileInfo(ApkName).LastWriteTime.ToString();
bool bNeedAPKInstall = true;
if (Params.IterativeDeploy)
{
// Check for apk installed with this package name on the device
IProcessResult InstalledResult = RunAdbCommand(Params, DeviceName, "shell pm list packages " + PackageName, null, ERunOptions.AppMustExist);
if (InstalledResult.Output.Contains(PackageName))
{
// See if apk is up to date on device
InstalledResult = RunAdbCommand(Params, DeviceName, "shell cat " + RemoteDir + "/APKFileStamp.txt", null, ERunOptions.AppMustExist);
if (InstalledResult.Output.StartsWith("APK: "))
{
if (InstalledResult.Output.Substring(5).Trim() == APKLastUpdateTime)
bNeedAPKInstall = false;
// Stop the previously running copy (uninstall/install did this before)
InstalledResult = RunAdbCommand(Params, DeviceName, "shell am force-stop " + PackageName, null, ERunOptions.AppMustExist);
if (InstalledResult.Output.Contains("Error"))
{
// force-stop not supported (Android < 3.0) so check if package is actually running
// Note: cannot use grep here since it may not be installed on device
InstalledResult = RunAdbCommand(Params, DeviceName, "shell ps", null, ERunOptions.AppMustExist);
if (InstalledResult.Output.Contains(PackageName))
{
// it is actually running so use the slow way to kill it (uninstall and reinstall)
bNeedAPKInstall = true;
}
}
}
}
}
// install new APK if needed
if (bNeedAPKInstall)
{
// try uninstalling an old app with the same identifier.
int SuccessCode = 0;
string UninstallCommandline = "uninstall " + PackageName;
RunAndLogAdbCommand(Params, DeviceName, UninstallCommandline, out SuccessCode);
// install the apk
string InstallCommandline = "install \"" + ApkName + "\"";
string InstallOutput = RunAndLogAdbCommand(Params, DeviceName, InstallCommandline, out SuccessCode);
int FailureIndex = InstallOutput.IndexOf("Failure");
// adb install doesn't always return an error code on failure, and instead prints "Failure", followed by an error code.
if (SuccessCode != 0 || FailureIndex != -1)
{
string ErrorMessage = string.Format("Installation of apk '{0}' failed", ApkName);
if (FailureIndex != -1)
{
string FailureString = InstallOutput.Substring(FailureIndex + 7).Trim();
if (FailureString != "")
{
ErrorMessage += ": " + FailureString;
}
}
if (ErrorMessage.Contains("OLDER_SDK"))
{
LogError("minSdkVersion is higher than Android version installed on device, possibly due to NDK API Level");
}
throw new AutomationException(ExitCode.Error_AppInstallFailed, ErrorMessage);
}
else
{
// giving EXTERNAL_STORAGE_WRITE permission to the apk for API23+
// without this permission apk can't access to the assets put into the device
string ReadPermissionCommandLine = "shell pm grant " + PackageName + " android.permission.READ_EXTERNAL_STORAGE";
string WritePermissionCommandLine = "shell pm grant " + PackageName + " android.permission.WRITE_EXTERNAL_STORAGE";
RunAndLogAdbCommand(Params, DeviceName, ReadPermissionCommandLine, out SuccessCode);
RunAndLogAdbCommand(Params, DeviceName, WritePermissionCommandLine, out SuccessCode);
}
}
// update the ue4commandline.txt
// update and deploy ue4commandline.txt
// always delete the existing commandline text file, so it doesn't reuse an old one
string IntermediateCmdLineFile = CombinePaths(SC.StageDirectory, "UE4CommandLine.txt");
Project.WriteStageCommandline(IntermediateCmdLineFile, Params, SC);
// copy files to device if we were staging
if (SC.Stage)
{
// cache some strings
string BaseCommandline = "push";
HashSet<string> EntriesToDeploy = new HashSet<string>();
if (Params.IterativeDeploy)
{
// always send UE4CommandLine.txt (it was written above after delta checks applied)
EntriesToDeploy.Add(IntermediateCmdLineFile);
// Add non UFS files if any to deploy
String NonUFSManifestPath = SC.GetNonUFSDeploymentDeltaPath(DeviceName);
if (File.Exists(NonUFSManifestPath))
{
string NonUFSFiles = File.ReadAllText(NonUFSManifestPath);
foreach (string Filename in NonUFSFiles.Split('\n'))
{
if (!string.IsNullOrEmpty(Filename) && !string.IsNullOrWhiteSpace(Filename))
{
EntriesToDeploy.Add(CombinePaths(SC.StageDirectory, Filename.Trim()));
}
}
}
// Add UFS files if any to deploy
String UFSManifestPath = SC.GetUFSDeploymentDeltaPath(DeviceName);
if (File.Exists(UFSManifestPath))
{
string UFSFiles = File.ReadAllText(UFSManifestPath);
foreach (string Filename in UFSFiles.Split('\n'))
{
if (!string.IsNullOrEmpty(Filename) && !string.IsNullOrWhiteSpace(Filename))
{
EntriesToDeploy.Add(CombinePaths(SC.StageDirectory, Filename.Trim()));
}
}
}
// For now, if too many files may be better to just push them all
if (EntriesToDeploy.Count > 500)
{
// make sure device is at a clean state
RunAdbCommand(Params, DeviceName, "shell rm -r " + RemoteDir);
EntriesToDeploy.Clear();
EntriesToDeploy.TrimExcess();
EntriesToDeploy.Add(SC.StageDirectory);
}
}
else
{
// make sure device is at a clean state
RunAdbCommand(Params, DeviceName, "shell rm -r " + RemoteDir);
// Copy UFS files..
string[] Files = Directory.GetFiles(SC.StageDirectory, "*", SearchOption.AllDirectories);
System.Array.Sort(Files);
// Find all the files we exclude from copying. And include
// the directories we need to individually copy.
HashSet<string> ExcludedFiles = new HashSet<string>();
SortedSet<string> IndividualCopyDirectories
= new SortedSet<string>((IComparer<string>)new LongestFirst());
foreach (string Filename in Files)
{
bool Exclude = false;
// Don't push the apk, we install it
Exclude |= Path.GetExtension(Filename).Equals(".apk", StringComparison.InvariantCultureIgnoreCase);
// For excluded files we add the parent dirs to our
// tracking of stuff to individually copy.
if (Exclude)
{
ExcludedFiles.Add(Filename);
// We include all directories up to the stage root in having
// to individually copy the files.
for (string FileDirectory = Path.GetDirectoryName(Filename);
!FileDirectory.Equals(SC.StageDirectory);
FileDirectory = Path.GetDirectoryName(FileDirectory))
{
if (!IndividualCopyDirectories.Contains(FileDirectory))
{
IndividualCopyDirectories.Add(FileDirectory);
}
}
if (!IndividualCopyDirectories.Contains(SC.StageDirectory))
{
IndividualCopyDirectories.Add(SC.StageDirectory);
}
}
}
// The directories are sorted above in "deepest" first. We can
// therefore start copying those individual dirs which will
// recreate the tree. As the subtrees will get copied at each
// possible individual level.
foreach (string DirectoryName in IndividualCopyDirectories)
{
string[] Entries
= Directory.GetFileSystemEntries(DirectoryName, "*", SearchOption.TopDirectoryOnly);
foreach (string Entry in Entries)
{
// We avoid excluded files and the individual copy dirs
// (the individual copy dirs will get handled as we iterate).
if (ExcludedFiles.Contains(Entry) || IndividualCopyDirectories.Contains(Entry))
{
continue;
}
else
{
EntriesToDeploy.Add(Entry);
}
}
}
if (EntriesToDeploy.Count == 0)
{
EntriesToDeploy.Add(SC.StageDirectory);
}
}
// We now have a minimal set of file & dir entries we need
// to deploy. Files we deploy will get individually copied
// and dirs will get the tree copies by default (that's
// what ADB does).
HashSet<IProcessResult> DeployCommands = new HashSet<IProcessResult>();
foreach (string Entry in EntriesToDeploy)
{
string FinalRemoteDir = RemoteDir;
string RemotePath = Entry.Replace(SC.StageDirectory, FinalRemoteDir).Replace("\\", "/");
string Commandline = string.Format("{0} \"{1}\" \"{2}\"", BaseCommandline, Entry, RemotePath);
// We run deploy commands in parallel to maximize the connection
// throughput.
DeployCommands.Add(
RunAdbCommand(Params, DeviceName, Commandline, null,
ERunOptions.Default | ERunOptions.NoWaitForExit));
// But we limit the parallel commands to avoid overwhelming
// memory resources.
if (DeployCommands.Count == DeployMaxParallelCommands)
{
while (DeployCommands.Count > DeployMaxParallelCommands / 2)
{
Thread.Sleep(1);
DeployCommands.RemoveWhere(
delegate (IProcessResult r)
{
return r.HasExited;
});
}
}
}
foreach (IProcessResult deploy_result in DeployCommands)
{
deploy_result.WaitForExit();
}
// delete the .obb file, since it will cause nothing we just deployed to be used
RunAdbCommand(Params, DeviceName, "shell rm " + DeviceObbName);
}
else if (SC.Archive)
{
// deploy the obb if there is one
string ObbPath = Path.Combine(SC.StageDirectory, GetFinalObbName(ApkName));
if (File.Exists(ObbPath))
{
// cache some strings
string BaseCommandline = "push";
string Commandline = string.Format("{0} \"{1}\" \"{2}\"", BaseCommandline, ObbPath, DeviceObbName);
RunAdbCommand(Params, DeviceName, Commandline);
}
}
else
{
// cache some strings
string BaseCommandline = "push";
string FinalRemoteDir = RemoteDir;
/*
// handle the special case of the UE4Commandline.txt when using content only game (UE4Game)
if (!Params.IsCodeBasedProject)
{
FinalRemoteDir = "/mnt/sdcard/UE4Game";
}
*/
string RemoteFilename = IntermediateCmdLineFile.Replace(SC.StageDirectory, FinalRemoteDir).Replace("\\", "/");
string Commandline = string.Format("{0} \"{1}\" \"{2}\"", BaseCommandline, IntermediateCmdLineFile, RemoteFilename);
RunAdbCommand(Params, DeviceName, Commandline);
}
// write new timestamp for APK (do it here since RemoteDir will now exist)
if (bNeedAPKInstall)
{
int SuccessCode = 0;
RunAndLogAdbCommand(Params, DeviceName, "shell \"echo 'APK: " + APKLastUpdateTime + "' > " + RemoteDir + "/APKFileStamp.txt\"", out SuccessCode);
}
}
}
/** Internal usage for GetPackageName */
private static string PackageLine = null;
private static Mutex PackageInfoMutex = new Mutex();
private static string LaunchableActivityLine = null;
/** Run an external exe (and capture the output), given the exe path and the commandline. */
private static string GetPackageInfo(string ApkName, bool bRetrieveVersionCode)
{
// we expect there to be one, so use the first one
string AaptPath = GetAaptPath();
PackageInfoMutex.WaitOne();
var ExeInfo = new ProcessStartInfo(AaptPath, "dump badging \"" + ApkName + "\"");
ExeInfo.UseShellExecute = false;
ExeInfo.RedirectStandardOutput = true;
using (var GameProcess = Process.Start(ExeInfo))
{
PackageLine = null;
LaunchableActivityLine = null;
GameProcess.BeginOutputReadLine();
GameProcess.OutputDataReceived += ParsePackageName;
GameProcess.WaitForExit();
}
PackageInfoMutex.ReleaseMutex();
string ReturnValue = null;
if (PackageLine != null)
{
// the line should look like: package: name='com.epicgames.qagame' versionCode='1' versionName='1.0'
string[] Tokens = PackageLine.Split("'".ToCharArray());
int TokenIndex = bRetrieveVersionCode ? 3 : 1;
if (Tokens.Length >= TokenIndex + 1)
{
ReturnValue = Tokens[TokenIndex];
}
}
return ReturnValue;
}
/** Returns the launch activity name to launch (must call GetPackageInfo first), returns "com.epicgames.ue4.SplashActivity" default if not found */
private static string GetLaunchableActivityName()
{
string ReturnValue = "com.epicgames.ue4.SplashActivity";
if (LaunchableActivityLine != null)
{
// the line should look like: launchable-activity: name='com.epicgames.ue4.SplashActivity' label='TappyChicken' icon=''
string[] Tokens = LaunchableActivityLine.Split("'".ToCharArray());
if (Tokens.Length >= 2)
{
ReturnValue = Tokens[1];
}
}
return ReturnValue;
}
/** Simple function to pipe output asynchronously */
private static void ParsePackageName(object Sender, DataReceivedEventArgs Event)
{
// DataReceivedEventHandler is fired with a null string when the output stream is closed. We don't want to
// print anything for that event.
if (!String.IsNullOrEmpty(Event.Data))
{
if (PackageLine == null)
{
string Line = Event.Data;
if (Line.StartsWith("package:"))
{
PackageLine = Line;
}
}
if (LaunchableActivityLine == null)
{
string Line = Event.Data;
if (Line.StartsWith("launchable-activity:"))
{
LaunchableActivityLine = Line;
}
}
}
}
static private string CachedAaptPath = null;
static private string LastAndroidHomePath = null;
private static uint GetRevisionValue(string VersionString)
{
// read up to 4 sections (ie. 20.0.3.5), first section most significant
// each section assumed to be 0 to 255 range
uint Value = 0;
try
{
string[] Sections= VersionString.Split(".".ToCharArray());
Value |= (Sections.Length > 0) ? (uint.Parse(Sections[0]) << 24) : 0;
Value |= (Sections.Length > 1) ? (uint.Parse(Sections[1]) << 16) : 0;
Value |= (Sections.Length > 2) ? (uint.Parse(Sections[2]) << 8) : 0;
Value |= (Sections.Length > 3) ? uint.Parse(Sections[3]) : 0;
}
catch (Exception)
{
// ignore poorly formed version
}
return Value;
}
private static string GetAaptPath()
{
// return cached path if ANDROID_HOME has not changed
string HomePath = Environment.ExpandEnvironmentVariables("%ANDROID_HOME%");
if (CachedAaptPath != null && LastAndroidHomePath == HomePath)
{
return CachedAaptPath;
}
// get a list of the directories in build-tools.. may be more than one set installed (or none which is bad)
string[] Subdirs = Directory.GetDirectories(Path.Combine(HomePath, "build-tools"));
if (Subdirs.Length == 0)
{
throw new AutomationException(ExitCode.Error_AndroidBuildToolsPathNotFound, "Failed to find %ANDROID_HOME%/build-tools subdirectory. Run SDK manager and install build-tools.");
}
// valid directories will have a source.properties with the Pkg.Revision (there is no guarantee we can use the directory name as revision)
string BestToolPath = null;
uint BestVersion = 0;
foreach (string CandidateDir in Subdirs)
{
string AaptFilename = Path.Combine(CandidateDir, Utils.IsRunningOnMono ? "aapt" : "aapt.exe");
uint RevisionValue = 0;
if (File.Exists(AaptFilename))
{
string SourcePropFilename = Path.Combine(CandidateDir, "source.properties");
if (File.Exists(SourcePropFilename))
{
string[] PropertyContents = File.ReadAllLines(SourcePropFilename);
foreach (string PropertyLine in PropertyContents)
{
if (PropertyLine.StartsWith("Pkg.Revision="))
{
RevisionValue = GetRevisionValue(PropertyLine.Substring(13));
break;
}
}
}
}
// remember it if newer version or haven't found one yet
if (RevisionValue > BestVersion || BestToolPath == null)
{
BestVersion = RevisionValue;
BestToolPath = AaptFilename;
}
}
if (BestToolPath == null)
{
throw new AutomationException(ExitCode.Error_AndroidBuildToolsPathNotFound, "Failed to find %ANDROID_HOME%/build-tools subdirectory with aapt. Run SDK manager and install build-tools.");
}
CachedAaptPath = BestToolPath;
LastAndroidHomePath = HomePath;
Log("Using this aapt: {0}", CachedAaptPath);
return CachedAaptPath;
}
private string GetBestDeviceArchitecture(ProjectParams Params, string DeviceName)
{
bool bMakeSeparateApks = UnrealBuildTool.UEDeployAndroid.ShouldMakeSeparateApks();
// if we are joining all .so's into a single .apk, there's no need to find the best one - there is no other one
if (!bMakeSeparateApks)
{
return "";
}
var AppArchitectures = new AndroidToolChain(Params.RawProjectPath).GetAllArchitectures();
// ask the device
IProcessResult ABIResult = RunAdbCommand(Params, DeviceName, " shell getprop ro.product.cpu.abi", null, ERunOptions.AppMustExist);
// the output is just the architecture
string DeviceArch = UnrealBuildTool.UEDeployAndroid.GetUE4Arch(ABIResult.Output.Trim());
// if the architecture wasn't built, look for a backup
if (!AppArchitectures.Contains(DeviceArch))
{
// go from 64 to 32-bit
if (DeviceArch == "-arm64")
{
DeviceArch = "-armv7";
}
// go from 64 to 32-bit
else if (DeviceArch == "-x64")
{
if (!AppArchitectures.Contains("-x86"))
{
DeviceArch = "-x86";
}
// if it didn't have 32-bit x86, look for 64-bit arm for emulation
// @todo android 64-bit: x86_64 most likely can't emulate arm64 at this ponit
// else if (Array.IndexOf(AppArchitectures, "-arm64") == -1)
// {
// DeviceArch = "-arm64";
// }
// finally try for 32-bit arm emulation (Houdini)
else
{
DeviceArch = "-armv7";
}
}
// use armv7 (with Houdini emulation)
else if (DeviceArch == "-x86")
{
DeviceArch = "-armv7";
}
else
{
// future-proof by dropping back to armv7 for unknown
DeviceArch = "-armv7";
}
}
// if after the fallbacks, we still don't have it, we can't continue
if (!AppArchitectures.Contains(DeviceArch))
{
throw new AutomationException(ExitCode.Error_NoApkSuitableForArchitecture, "Unable to run because you don't have an apk that is usable on {0}. Looked for {1}", DeviceName, DeviceArch);
}
return DeviceArch;
}
private string GetBestGPUArchitecture(ProjectParams Params, string DeviceName)
{
bool bMakeSeparateApks = UnrealBuildTool.UEDeployAndroid.ShouldMakeSeparateApks();
// if we are joining all .so's into a single .apk, there's no need to find the best one - there is no other one
if (!bMakeSeparateApks)
{
return "";
}
var AppGPUArchitectures = new AndroidToolChain(Params.RawProjectPath).GetAllGPUArchitectures();
// get the device extensions
IProcessResult ExtensionsResult = RunAdbCommand(Params, DeviceName, "shell dumpsys SurfaceFlinger", null, ERunOptions.AppMustExist);
string Extensions = ExtensionsResult.Output.Trim();
// look for AEP support (on device and in project)
if (Extensions.Contains("GL_ANDROID_extension_pack_es31a") && Extensions.Contains("GL_EXT_color_buffer_half_float"))
{
if (AppGPUArchitectures.Contains("-esdeferred"))
{
return "-esdeferred";
}
}
return "-es2";
}
public override IProcessResult RunClient(ERunOptions ClientRunFlags, string ClientApp, string ClientCmdLine, ProjectParams Params)
{
//make a copy of the device names, we'll be working through them
List<string> DeviceNames = new List<string>();
//same with the package names
List<string> PackageNames = new List<string>();
foreach (string DeviceName in Params.DeviceNames)
{
//save the device name
DeviceNames.Add(DeviceName);
//get the package name and save that
string DeviceArchitecture = GetBestDeviceArchitecture(Params, DeviceName);
string GPUArchitecture = GetBestGPUArchitecture(Params, DeviceName);
string ApkName = ClientApp + DeviceArchitecture + ".apk";
if (!File.Exists(ApkName))
{
ApkName = GetFinalApkName(Params, Path.GetFileNameWithoutExtension(ClientApp), true, DeviceArchitecture, GPUArchitecture);
}
Console.WriteLine("Apk='{0}', ClientApp='{1}', ExeName='{2}'", ApkName, ClientApp, Params.ProjectGameExeFilename);
// run aapt to get the name of the intent
string PackageName = GetPackageInfo(ApkName, false);
if (PackageName == null)
{
throw new AutomationException(ExitCode.Error_FailureGettingPackageInfo, "Failed to get package name from " + ClientApp);
}
PackageNames.Add(PackageName);
// clear the log for the device
RunAdbCommand(Params, DeviceName, "logcat -c");
// start the app on device!
string CommandLine = "shell am start -n " + PackageName + "/" + GetLaunchableActivityName();
RunAdbCommand(Params, DeviceName, CommandLine, null, ClientRunFlags);
// save the output to the staging directory
string LogPath = Path.Combine(Params.BaseStageDirectory, "Android\\logs");
Directory.CreateDirectory(LogPath);
}
//now check if each device still has the game running, and time out if it's taking too long
DateTime StartTime = DateTime.Now;
int TimeOutSeconds = Params.RunTimeoutSeconds;
while (DeviceNames.Count > 0)
{
for(int DeviceIndex = 0; DeviceIndex < DeviceNames.Count; DeviceIndex++)
{
string DeviceName = DeviceNames[DeviceIndex];
//replace the port name in the case of deploy while adb is using wifi
string SanitizedDeviceName = DeviceName.Replace(":", "_");
bool FinishedRunning = false;
IProcessResult ProcessesResult = RunAdbCommand(Params, DeviceName, "shell ps", null, ERunOptions.SpewIsVerbose);
string RunningProcessList = ProcessesResult.Output;
if (!RunningProcessList.Contains(PackageNames[DeviceIndex]))
{
FinishedRunning = true;
}
Thread.Sleep(10);
if(!FinishedRunning)
{
TimeSpan DeltaRunTime = DateTime.Now - StartTime;
if ((DeltaRunTime.TotalSeconds > TimeOutSeconds) && (TimeOutSeconds != 0))
{
Log("Device: " + DeviceName + " timed out while waiting for run to finish");
FinishedRunning = true;
}
}
//log the results, then clear out the device from our list
if(FinishedRunning)
{
// this is just to get the ue4 log to go to the output
RunAdbCommand(Params, DeviceName, "logcat -d -s UE4 -s Debug");
// get the log we actually want to save
IProcessResult LogFileProcess = RunAdbCommand(Params, DeviceName, "logcat -d", null, ERunOptions.AppMustExist);
string LogPath = Path.Combine(Params.BaseStageDirectory, "Android\\logs");
string LogFilename = Path.Combine(LogPath, "devicelog" + SanitizedDeviceName + ".log");
string ServerLogFilename = Path.Combine(CmdEnv.LogFolder, "devicelog" + SanitizedDeviceName + ".log");
File.WriteAllText(LogFilename, LogFileProcess.Output);
File.WriteAllText(ServerLogFilename, LogFileProcess.Output);
DeviceNames.RemoveAt(DeviceIndex);
PackageNames.RemoveAt(DeviceIndex);
--DeviceIndex;
}
}
}
return null;
}
public override void GetFilesToDeployOrStage(ProjectParams Params, DeploymentContext SC)
{
// Add any Android shader cache files
string ProjectShaderDir = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(Params.RawProjectPath.ToString())), "Build/ShaderCaches/Android");
SC.StageFiles(StagedFileType.UFS, ProjectShaderDir, "*.*", true, null, null, true);
}
/// <summary>
/// Gets cook platform name for this platform.
/// </summary>
/// <returns>Cook platform string.</returns>
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android";
}
public override bool DeployPakInternalLowerCaseFilenames()
{
return false;
}
public override bool DeployLowerCaseFilenames(bool bUFSFile)
{
return false;
}
public override string LocalPathToTargetPath(string LocalPath, string LocalRoot)
{
return LocalPath.Replace("\\", "/").Replace(LocalRoot, "../../..");
}
public override bool IsSupported { get { return true; } }
public override string Remap(string Dest)
{
return Dest;
}
public override PakType RequiresPak(ProjectParams Params)
{
// if packaging is enabled, always create a pak, otherwise use the Params.Pak value
return Params.Package ? PakType.Always : PakType.DontCare;
}
public override bool SupportsMultiDeviceDeploy
{
get
{
return true;
}
}
/*
public override bool RequiresPackageToDeploy
{
get { return true; }
}
*/
public override List<string> GetDebugFileExtentions()
{
return new List<string> { };
}
}
public class AndroidPlatformMulti : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_Multi";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "Multi");
}
}
public class AndroidPlatformATC : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_ATC";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "ATC");
}
}
public class AndroidPlatformDXT : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_DXT";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "DXT");
}
}
public class AndroidPlatformETC1 : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_ETC1";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "ETC1");
}
}
public class AndroidPlatformETC2 : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_ETC2";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "ETC2");
}
}
public class AndroidPlatformPVRTC : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_PVRTC";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "PVRTC");
}
}
public class AndroidPlatformASTC : AndroidPlatform
{
public override string GetCookPlatform(bool bDedicatedServer, bool bIsClientOnly)
{
return "Android_ASTC";
}
public override TargetPlatformDescriptor GetTargetPlatformDescriptor()
{
return new TargetPlatformDescriptor(TargetPlatformType, "ASTC");
}
}