You've already forked UnrealEngineUWP
mirror of
https://github.com/izzy2lost/UnrealEngineUWP.git
synced 2026-03-26 18:15:20 -07:00
a287480c36
Most UE4 platforms use a 2-byte TCHAR, however some still use a 4-byte TCHAR. The platforms that use a 4-byte TCHAR expect their string data to be UTF-32, however there are parts of UE4 that serialize FString data as a series of UCS2CHAR, simply narrowing or widening each TCHAR in turn. This can result in invalid or corrupted UTF-32 strings (either UTF-32 strings containing UTF-16 surrogates, or UTF-32 code points that have been truncated to 2-bytes), which leads to either odd behavior or crashes. This change updates the parts of UE4 that process FString data as a series of 2-byte values to do so on the correct UTF-16 interpretation of the data, converting to/from UTF-32 as required on platforms that use a 4-byte TCHAR. This conversion is a no-op on platforms that use a 2-byte TCHAR as the string is already assumed to be valid UTF-16 data. It should also be noted that while FString may contain UTF-16 code units on platforms using a 2-byte TCHAR, this change doesn't do anything to make FString represent a Unicode string on those platforms (ie, a string that understands and works on code points), but is rather just a bag of code units. Two new variable-width string converters have be added to facilitate the conversion (modelled after the TCHAR<->UTF-8 converters), TUTF16ToUTF32_Convert and TUTF32ToUTF16_Convert. These are used for both TCHAR<->UTF16CHAR conversion when needed, but also for TCHAR<->wchar_t conversion on platforms that use char16_t for TCHAR along with having a 4-byte wchar_t (as defined by the new PLATFORM_WCHAR_IS_4_BYTES option). These conversion routines are accessed either via the conversion macros (TCHAR_TO_UTF16, UTF16_TO_TCHAR, TCHAR_TO_WCHAR, and WCHAR_TO_TCHAR), or by using a conversion struct (FTCHARToUTF16, FUTF16ToTCHAR, FTCHARToWChar, and FWCharToTCHAR), which is the same pattern as the existing TCHAR<->UTF-8 conversion. Both the macros and the structs are defined as no-ops when the conversion isn't needed, but always exist so that code can be written in a portable way. Very little code actually needed updating to use UTF-16, as the vast majority makes no assumptions about the size of TCHAR, nor how FString should be serialized. The main places were the FString archive serialization and the JSON reader/writer, along with some minor fixes to the UTF-8 conversion logic for platforms using a 4-byte TCHAR. Tests have been added to verify that an FString representing a UTF-32 code point can be losslessly converted to/from UTF-8 and UTF-16, and serialized to/from an archive. #jira #rb Steve.Robb, Josh.Adams #ROBOMERGE-SOURCE: CL 8676728 via CL 8687863 #ROBOMERGE-BOT: (v421-8677696) [CL 8688048 by jamie dale in Main branch]
61 lines
2.0 KiB
C++
61 lines
2.0 KiB
C++
// Copyright 1998-2019 Epic Games, Inc. All Rights Reserved.
|
|
|
|
#include "MobileJSStructSerializerBackend.h"
|
|
|
|
#if PLATFORM_ANDROID || PLATFORM_IOS
|
|
|
|
#include "MobileJSScripting.h"
|
|
#include "UObject/UnrealType.h"
|
|
#include "UObject/PropertyPortFlags.h"
|
|
#include "Templates/Casts.h"
|
|
|
|
void FMobileJSStructSerializerBackend::WriteProperty(const FStructSerializerState& State, int32 ArrayIndex)
|
|
{
|
|
// The parent class serialzes UObjects as NULLs
|
|
if (State.ValueType == UObjectProperty::StaticClass())
|
|
{
|
|
WriteUObject(State, CastChecked<UObjectProperty>(State.ValueProperty)->GetPropertyValue_InContainer(State.ValueData, ArrayIndex));
|
|
}
|
|
// basic property type (json serializable)
|
|
else
|
|
{
|
|
FJsonStructSerializerBackend::WriteProperty(State, ArrayIndex);
|
|
}
|
|
}
|
|
|
|
void FMobileJSStructSerializerBackend::WriteUObject(const FStructSerializerState& State, UObject* Value)
|
|
{
|
|
// Note this function uses WriteRawJSONValue to append non-json data to the output stream.
|
|
FString RawValue = Scripting->ConvertObject(Value);
|
|
if ((State.ValueProperty == nullptr) || (State.ValueProperty->ArrayDim > 1) || (State.ValueProperty->GetOuter()->GetClass() == UArrayProperty::StaticClass()))
|
|
{
|
|
GetWriter()->WriteRawJSONValue(RawValue);
|
|
}
|
|
else if (State.KeyProperty != nullptr)
|
|
{
|
|
FString KeyString;
|
|
State.KeyProperty->ExportTextItem(KeyString, State.KeyData, nullptr, nullptr, PPF_None);
|
|
GetWriter()->WriteRawJSONValue(KeyString, RawValue);
|
|
}
|
|
else
|
|
{
|
|
GetWriter()->WriteRawJSONValue(Scripting->GetBindingName(State.ValueProperty), RawValue);
|
|
}
|
|
}
|
|
|
|
FString FMobileJSStructSerializerBackend::ToString()
|
|
{
|
|
ReturnBuffer.Add(0);
|
|
ReturnBuffer.Add(0); // Add two as we're dealing with UTF-16, so 2 bytes
|
|
return UTF16_TO_TCHAR((UTF16CHAR*)ReturnBuffer.GetData());
|
|
}
|
|
|
|
FMobileJSStructSerializerBackend::FMobileJSStructSerializerBackend(TSharedRef<class FMobileJSScripting> InScripting)
|
|
: FJsonStructSerializerBackend(Writer, EStructSerializerBackendFlags::Legacy)
|
|
, Scripting(InScripting)
|
|
, ReturnBuffer()
|
|
, Writer(ReturnBuffer)
|
|
{
|
|
}
|
|
|
|
#endif // PLATFORM_ANDROID || PLATFORM_IOS
|