PRACTICAL GUIDEUnder review

UTF-8, ANSI and special-character encoding

Identifies byte encoding before converting text.

UTF-8, ANSI and special-character encodingAdvPL

01 · OVERVIEW

Overview

ANSI does not identify one universal code page. Identify source bytes and destination: EncodeUTF8 converts a known code page to UTF-8; DecodeUTF8 converts to the requested page, with documented cp1252 default. Converting already converted text again can corrupt it. EncodeUTF8 can return Nil for bytes invalid in the declared page. The example constructs the cp1252 byte for é using Chr(233), avoiding source-file encoding dependence, and checks a round trip; it does not assume every Unicode character fits cp1252. (TOTVS, n.d., refs. 1–2)

03 · PRACTICAL EXAMPLE

Commented example

#Include "Protheus.ch"
User Function Ed304()
    Local cSource := Chr(233)
    Local cUtf8 := EncodeUTF8(cSource, "cp1252")
    // Check type before using the conversion.
    If ValType(cUtf8) == "C"
        ConOut(cValToChar(DecodeUTF8(cUtf8, "cp1252") == cSource))
    EndIf
Return Nil

Expected result

True equality is expected for this valid byte; investigate missing output as conversion failure. Under review: editorial example neither compiled nor run in Protheus. Check includes, release, RDD and context before testing.

BEST PRACTICES

Separate source, transport and storage encodings. Do not repair accents using ad hoc replacements or hide invalid data by suppressing conversion errors.
Under review: editorial example neither compiled nor run in Protheus. Check includes, release, RDD and context before testing.

UTF-8, ANSI and special-character encoding

Overview

Commented example

Related content

Comments