!z      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ Safe &'-.4=?HUVXtyped-encodingType safety over encodings makes decoding process safe. However failures are still possible due to bugs or unsafe payload modifications. UnexpectedDecodeEx represents such errors.typed-encodingRepresents errors in encodingtyped-encoding<Represents errors in recovery (recreation of encoded types).typed-encodingconstructor is to be treated as Unsafe to Encode and Decode instance implementations particular encoding instances may expose smart constructors for limited data types  Safe-.=>?@AHUVXtyped-encodingHuman friendly version of Show typed-encodingERecovery errors are expected unless Recovery allows Identity instance"typed-encodingWith type safety in pace decoding errors should be unexpected this class can be used to provide extra info if decoding could fail$typed-encoding/Polymorphic data payloads used to encode/decode*typed-encoding'TODO use singletons definition instead?.typed-encoding=Used to safely recover encoded data validating all encodingss=typed-encoding5Unsafe implementation guarded by safe type definitionMtyped-encoding*displ (Proxy :: Proxy ["FIRST", "SECOND"])"FIRST,SECOND"$ !"#$%&'()*+-,./0123456789:;<=>?@A$6745823019./+-,:*;<=>()&'$%"#? !@A Safe B=  !"#$%&'()*+,-./0123456789:;<=>?@ASafe.=>?@AHUVX!TUVTUVSafe.@A"J/  !"#$%&'()*+,-./0123456789:;<=>?@A  Safe.=>?@AUV5 dtyped-encodingType-safer version of Byte-string to text conversion that prevent invalid UTF8 bytestrings to be conversted to B64 encoded Text.ftyped-encodingConverts encoded text to ByteString adding "r-UTF8" annotation. The question is why "r-UTF8", not for example, "r-UTF16"? No reason, there maybe a diffrent combinator for that in the future or one that accepts a proxy.htyped-encoding|B64 encoded bytestring can be converted to Text as "enc-B64-nontext" preventing it from being B64-decoded directly to Text{typed-encodingEffectful instance for corruption detection. This protocol is used, for example, in emails. It is a well known encoding and hackers will have no problem making undetectable changes, but error handling at this stage could verify that email was corrupted.~typed-encodingCallow to treat B64 encodings as ASCII forgetting about B64 encodingZlet tstB64 = encodeAll . toEncoding () $ "Hello World" :: Enc '["enc-B64"] () B.ByteStringWdispl (flattenAs (Proxy :: Proxy "r-ASCII") tstB64 :: Enc '["r-ASCII"] () B.ByteString)3"MkEnc '[r-ASCII] () (ByteString SGVsbG8gV29ybGQ=)" defghijklmn defghijklmnSafe5Safe &'-.4=?HUV7typed-encoding_Allows to operate within Enc. These are considered unsafe. keeping the same list of encodings Safe-.=>?@AHUVXhRtyped-encodingempty string is valid utf8typed-encodingType-safer version of Data.Text.Encoding.encodeUtf8;displ $ text2ByteStringS $ toEncoding () ("text" :: T.Text)&"MkEnc '[r-UTF8] () (ByteString text)"typed-encoding3Type-safer version of Data.Text.Encoding.decodeUtf8olet Right tst = encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)displ $ byteString2TextS tst!"MkEnc '[] () (Text Hello World)"typed-encodingIdentity property "byteString2TextS . text2ByteStringS == id" prop> t -> t == (fromEncoding . txtBsSIdProp (Proxy :: Proxy '[]) . toEncoding () $ t)typed-encoding>Identity property "text2ByteStringS . byteString2TextS == id".V\(t :: Enc '["r-UTF8"] () B.ByteString) -> t == (bsTxtIdProp (Proxy :: Proxy '[]) $ t)typed-encodinghelper function checks that given ByteString, if is encoded as Left is must be not Utf8 decodable is is encoded as Right is must be Utf8 encodable typed-encodingYUTF8 encodings are defined for ByteString only as that would not make much sense for Text\encodeFAll . toEncoding () $ "\xc3\xb1" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)!Right (MkEnc Proxy () "\195\177")\encodeFAll . toEncoding () $ "\xc3\x28" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)rLeft (EncodeEx "r-UTF8" (Cannot decode byte '\xc3': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream))Following test uses Q helper that checks that bytes are encoded as Right iff they are valid UTF8 bytes\(b :: B.ByteString) -> verEncoding b (fmap (fromEncoding . decodeAll . proxiedId (Proxy :: Proxy (Enc '["r-UTF8"] _ _))) . (encodeFAll :: _ -> Either EncodeEx _). toEncoding () $ b) Safe .=>?@AUVXtyped-encodingDallow to treat ASCII encodings as UTF8 forgetting about B64 encodingolet Right tstAscii = encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-ASCII"] () T.Text)1displ (inject (Proxy :: Proxy "r-UTF8") tstAscii)'"MkEnc '[r-UTF8] () (Text Hello World)" Safe .=>?@AUV typed-encodingExample value to play with`encodeFAll . toEncoding () $ "HeLlo world" :: Either EncodeEx (Enc '["r-ASCII"] () B.ByteString)$Right (MkEnc Proxy () "HeLlo world")typed-encodingabove with either removedtyped-encoding3When converted to Text the annotation is preserved.DCurrently separate function is defined for each allowed conversion. ,displ $ EnASCII.byteString2TextS helloAsciiB("MkEnc '[r-ASCII] () (Text HeLlo world)"typed-encoding/To get UTF8 annotation, instead of doing this: _encodeFAll . toEncoding () $ "HeLlo world" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)$Right (MkEnc Proxy () "HeLlo world")/We should be able to convert the ASCII version.This is done using ( typeclass.inject1 method accepts proxy to specify superset to use.4displ $ inject (Proxy :: Proxy "r-UTF8") helloAsciiB-"MkEnc '[r-UTF8] () (ByteString HeLlo world)"typed-encoding$We put Base64 on the UFT8 ByteString;displ $ encodePart (Proxy :: Proxy '["enc-B64"]) helloUtf8B:"MkEnc '[enc-B64,r-UTF8] () (ByteString SGVMbG8gd29ybGQ=)"typed-encoding\.. and copy it over to Text. but UTF8 would be redundant in Text so the "r-UTF8" is dropped':t EnB64.byteString2TextS helloUtf8B64BBEnB64.byteString2TextS helloUtf8B64B :: Enc '["enc-B64"] () T.TextConversely moving back to ByteString recovers the annotation. (there could be a choice of a UTF annotation to recover in the future)':t EnB64.text2ByteStringS helloUtf8B64T$EnB64.text2ByteStringS helloUtf8B64T1... :: Enc '["enc-B64", "r-UTF8"] () B.ByteStringtyped-encoding7 a binary, one that does not even represent valid UTF8.JencodeAll . toEncoding () $ "\195\177" :: Enc '["enc-B64"] () B.ByteStringMkEnc Proxy () "w7E="hI is a fuction that allows to convert Base 64 ByteString that is not UTF8.#:t EnB64.byteString2TextS' notTextB EnB64.byteString2TextS' notTextB)... :: Enc '["enc-B64-nontext"] () T.TextOThe result is annotated as "enc-B64-nontext" which prevents decoding it within < type. We can only move it back to ByteString as "enc-B64".typed-encodingOrecreateAll . toEncoding () $ "abc==CB" :: Enc '["enc-B64-len"] () B.ByteStringMkEnc Proxy () "abc==CB"The rest of Haskell does lenient decoding, type safety allows this library to use it for recovery. lenient algorithms are not partial and automatically fix invalid input:`recreateFAll . toEncoding () $ "abc==CB" :: Either RecreateEx (Enc '["enc-B64"] () B.ByteString)/Left (RecreateEx "enc-B64" ("invalid padding"))QThis library allows to recover to "enc-B64-len" which is different than "enc-B64"l- allows to convert "enc-B64-len" to "enc-B64"-displ $ EnB64.acceptLenientS lenientSomething'"MkEnc '[enc-B64] () (ByteString abc=)"!This is now properly encoded data]recreateFAll . toEncoding () $ "abc=" :: Either RecreateEx (Enc '["enc-B64"] () B.ByteString)Right (MkEnc Proxy () "abc=")&Except the content could be surprising1decodeAll $ EnB64.acceptLenientS lenientSomethingMkEnc Proxy () "i\183"typed-encodingBase 64 encodes binary data as ASCII text. thus, we should be able to treat "enc-B64" as "r-ASCII" losing some information. this is done using & type class5:t flattenAs (Proxy :: Proxy "r-ASCII") helloUtf8B64B2flattenAs (Proxy :: Proxy "r-ASCII") helloUtf8B64B'... :: Enc '["r-ASCII"] () B.ByteString Safe .=>?@AUVXtyped-encoding6encoding function, typically should be module private typed-encoding,dual purpose decoding and recovery function.(This typically should be module private.decodeSign "3:abc" Right "abc"decodeSign "4:abc"Left "Corrupted Signature"typed-encodingEncoded hello world example. helloSignedMkEnc Proxy () "11:Hello World"&fromEncoding . decodeAll $ helloSigned "Hello World"typed-encodingproperty checks that = values are exected to decode without error after encoding.\t -> propEncDectyped-encodingJHacker example The data was transmitted over a network and got corrupted.0let payload = getPayload $ helloSigned :: T.Text%let newpay = payload <> " corruption"WrecreateFAll . toEncoding () $ newpay :: Either RecreateEx (Enc '["my-sign"] () T.Text)3Left (RecreateEx "my-sign" ("Corrupted Signature"))XrecreateFAll . toEncoding () $ payload :: Either RecreateEx (Enc '["my-sign"] () T.Text)'Right (MkEnc Proxy () "11:Hello World")typed-encodingRecreation allows effectful f? to check for tampering with data. Implementation simply uses  % combinator on the recovery function.typed-encodingDecoding allows effectful f9 to allow for troubleshooting and unsafe payload changes.Implementation simply uses   combinator on the ?# composed with decoding function. " has Identity instance allowing for decoding that assumes errors are not possible. For debugging purposes or when unsafe changes to "my-sign" Error UnexpectedDecodeEx instance can be used.typed-encodingdBecause encoding function is pure we can create instance of EncodeF that is polymorphic in effect f. This is done using  combinator. Safe.@Aktyped-encodingExample configurationtyped-encoding"Hello World" encoded as Base64helloB64!MkEnc Proxy () "SGVsbG8gV29ybGQ="displ helloB643"MkEnc '[enc-B64] () (ByteString SGVsbG8gV29ybGQ=)"MencodeAll . toEncoding () $ "Hello World" :: Enc '["enc-B64"] () B.ByteString!MkEnc Proxy () "SGVsbG8gV29ybGQ="typed-encoding!Previous text decoded from Base64#fromEncoding . decodeAll $ helloB64 "Hello World"typed-encoding- allows for recovering data at program boundaries (for example, when parsing JSON input). It makes sure that the content satisfies specified encodings.irecreateFAll . toEncoding () $ "SGVsbG8gV29ybGQ=" :: Either RecreateEx (Enc '["enc-B64"] () B.ByteString))Right (MkEnc Proxy () "SGVsbG8gV29ybGQ=")hrecreateFAll . toEncoding () $ "SGVsbG8gV29ybGQ" :: Either RecreateEx (Enc '["enc-B64"] () B.ByteString)/Left (RecreateEx "enc-B64" ("invalid padding"))typed-encodingx"Hello World" double-Base64 encoded. Notice the same code used as in single encoding, the game is played at type level.WencodeAll . toEncoding () $ "Hello World" :: Enc '["enc-B64","enc-B64"] () B.ByteString)MkEnc Proxy () "U0dWc2JHOGdWMjl5YkdRPQ=="displ helloB64B64C"MkEnc '[enc-B64,enc-B64] () (ByteString U0dWc2JHOGdWMjl5YkdRPQ==)"typed-encodingFDouble Base64 encoded "Hello World" with one layer of encoding removedZdecodePart (Proxy :: Proxy '["enc-B64"]) $ helloB64B64 :: Enc '["enc-B64"] () B.ByteString!MkEnc Proxy () "SGVsbG8gV29ybGQ="!helloB64B64PartDecode == helloB64Truetyped-encoding all the way to *Notice a similar polymorphism in decoding.6fromEncoding . decodeAll $ helloB64B64 :: B.ByteString "Hello World""We can also decode all the parts: OfromEncoding . decodePart (Proxy :: Proxy '["enc-B64","enc-B64"]) $ helloB64B64 "Hello World"typed-encoding9what happens when we try to recover encoded once text to Enc '["enc-B64", "enc-B64"]. BAgain, notice the same expression is used as in previous recovery.trecreateFAll . toEncoding () $ "SGVsbG8gV29ybGQ=" :: Either RecreateEx (Enc '["enc-B64", "enc-B64"] () B.ByteString)/Left (RecreateEx "enc-B64" ("invalid padding"))typed-encoding"do-UPPER" (from * module) encoding applied to "Hello World"Notice a namespace thing going on, "enc-" is encoding, "do-" is some transformation. These are typically not reversible, some could be recoverable.FThe same code is used as in "enc-" examples to encode (now transform).HencodeAll . toEncoding () $ "Hello World" :: Enc '["do-UPPER"] () T.TextMkEnc Proxy () "HELLO WORLD"typed-encodingSample compound transformation VencodeAll . toEncoding () $ "HeLLo world" :: Enc '["do-reverse", "do-Title"] () T.TextMkEnc Proxy () "dlroW olleH" typed-encoding is needed in following examplestyped-encoding9Configuration can be used to impact the encoding process.So far we had used () as configuration of all encodings. But since both "do-reverse", "do-Title" are polymorphic in configuration we can also do this:cencodeAll . toEncoding exampleConf $ "HeLLo world" :: Enc '["do-reverse", "do-Title"] Config T.TextLMkEnc Proxy (Config {sizeLimit = SizeLimit {unSizeLimit = 8}}) "dlroW olleH"tencodeAll . toEncoding exampleConf $ "HeLlo world" :: Enc '["do-size-limit", "do-reverse", "do-Title"] Config T.TextIMkEnc Proxy (Config {sizeLimit = SizeLimit {unSizeLimit = 8}}) "dlroW ol"#Instead, encode previously defined & by reversing it and adding size limitencodePart (Proxy :: Proxy '["do-size-limit", "do-reverse"]) helloTitle :: Enc '["do-size-limit", "do-reverse", "do-Title"] Config T.TextIMkEnc Proxy (Config {sizeLimit = SizeLimit {unSizeLimit = 8}}) "dlroW ol"typed-encoding#... and we unwrap the B64 part only8decodePart (Proxy :: Proxy '["enc-B64"]) $ helloLimitB64IMkEnc Proxy (Config {sizeLimit = SizeLimit {unSizeLimit = 8}}) "HeLlo wo"typed-encoding4ASCII char set ByteStrings are sequences of Bytes (Q). The type is very permissive, it may contain binary data such as jpeg picture."r-ASCII" encoding acts as partial identity function it does not change any bytes in bytestring but it fails if a byte is outside of ASCII range (in Either monad).GNote naming thing: "r-" is partial identity ("r-" is from restriction).`encodeFAll . toEncoding () $ "HeLlo world" :: Either EncodeEx (Enc '["r-ASCII"] () B.ByteString)$Right (MkEnc Proxy () "HeLlo world")typed-encodingUArguably the type we used for helloB64 was too permissive. a better version is here:kencodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["enc-B64", "r-ASCII"] () B.ByteString)*Right (MkEnc Proxy () "SGVsbG8gV29ybGQ=") typed-encoding:decodePart (Proxy :: Proxy '["enc-B64"]) <$> helloAsciiB64$Right (MkEnc Proxy () "Hello World") Safe .=>?@AUVStyped-encodingStarting exampletyped-encodingwith either removedtyped-encoding-- is the way to recover encoding in a safe way!let payload = getPayload exAsciiT/let newPayload = payload <> " some extra stuff"[recreateFAll . toEncoding () $ newPayload :: Either RecreateEx (Enc '["r-ASCII"] () T.Text)/Right (MkEnc Proxy () "HELLO some extra stuff")typed-encodingThe issue with - is that it may be expensive.This apprach uses B to perform (in general risky) operation on the internal payload. exAsciiTERight (MkEnc Proxy () "HELLO")7exAsciiTE >>= pure . Unsafe.withUnsafe (fmap T.toLower)Right (MkEnc Proxy () "hello")Example uses of H within encoded data this operation is safe for ASCII restriction but Enc '["r-ASCII"] () T.TextV does not expose it We use Functor instance of Unsafe wrapper type to accomplish thistyped-encoding-Similar example uses applicative instance of let Right hELLO = exAsciiTElet Right hello = toLowerAsciiOdispl $ Unsafe.runUnsafe ((<>) <$> Unsafe.Unsafe hELLO <*> Unsafe.Unsafe hello)'"MkEnc '[r-ASCII] () (Text HELLOhello)"Safe(Safe !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefgghijklmnopqrstuvwxyz{|}~pijklxvywpvwxyp p i   -typed-encoding-0.1.0.0-AuPwLeB2Ex7Jl4MzEB6XAy!Data.TypedEncoding.Internal.Types!Data.TypedEncoding.Internal.Class*Data.TypedEncoding.Instances.Encode.Sample#Data.TypedEncoding.Instances.Base64!Data.TypedEncoding.Internal.UtilsData.TypedEncoding.Unsafe!Data.TypedEncoding.Instances.UTF8"Data.TypedEncoding.Instances.ASCII"Examples.TypedEncoding.Conversions&Examples.TypedEncoding.DiySignEncodingExamples.TypedEncoding.OverviewExamples.TypedEncoding.Unsafe$Data.TypedEncoding.Instances.SupportData.TypedEncoding#Data.TypedEncoding.Instances.EncodeSample Data.WordWord8UnsafeExamples.TypedEncodingPaths_typed_encodingUnexpectedDecodeExEncodeEx RecreateExEncMkEnc toEncoding fromEncoding implTranF implEncodeF implDecodeFimplCheckPrevF implTranF' implEncodeF' implDecodeF' implTranP implEncodeP implTranP' implEncodeP' getPayloadunsafeSetPayloadwithUnsafeCoerceunsafeChangePayload$fShowRecreateEx$fShowEncodeEx$fShowUnexpectedDecodeEx $fShowEnc$fEqEncDispldispl RecreateErr recoveryErrUnexpectedDecodeErrunexpectedDecodeErrHasAhas FlattenAs flattenAsSubsetinjectAppend RecreateFAll checkFAll recreateFAll RecreateF checkPrevF DecodeFAll decodeFAllDecodeFdecodeF EncodeFAll encodeFAllEncodeFencodeF encodeAll decodeAll recreateAll encodeFPart encodePart decodeFPart decodePart asUnexpected asRecreateErr errorOnLeft$fEncodeFAllkf:cstr$fEncodeFAllkf[]cstr$fDecodeFAllkf:cstr$fDecodeFAllkf[]cstr$fRecreateFAllkf:cstr$fRecreateFAllkf[]cstr $fHasAa()$fUnexpectedDecodeErrTYPEEither!$fUnexpectedDecodeErrTYPEIdentity$fRecreateErrTYPEEither $fDisplEnc $fDisplProxy $fDisplProxy0$fDisplByteString$fDisplByteString0 $fDisplText $fDisplText0 $fDispl[] SizeLimit unSizeLimit$fEncodeFTYPEfEncEnc$fEncodeFTYPEfEncEnc0$fEncodeFTYPEfEncEnc1$fEncodeFTYPEfEncEnc2$fEncodeFTYPEfEncEnc3$fEncodeFTYPEfEncEnc4$fEncodeFTYPEfEncEnc5$fRecreateFTYPEfEncEnc$fEncodeFTYPEfEncEnc6$fEncodeFTYPEfEncEnc7$fEncodeFTYPEfEncEnc8 $fEqSizeLimit$fShowSizeLimitbyteString2TextSbyteString2TextLtext2ByteStringStext2ByteStringLbyteString2TextS'byteString2TextL'text2ByteStringS'text2ByteStringL'acceptLenientSacceptLenientLprxyB64$fDecodeFTYPEfEncEnc$fRecreateFTYPEfEncEnc0$fDecodeFTYPEfEncEnc0$fRecreateFTYPEfEncEnc1$fRecreateFTYPEfEncEnc2$fDecodeFTYPEfEncEnc1$fRecreateFTYPEfEncEnc3$fRecreateFTYPEfEncEnc4$fDecodeFTYPEfEncEnc2#$fFlattenAsSymbol"enc-B64""r-ASCII"+$fFlattenAsSymbol"enc-B64-nontext""r-ASCII" explainBool proxiedId runUnsafe withUnsafe $fMonadUnsafe$fApplicativeUnsafe$fFunctorUnsafe $fShowUnsafe emptyUTF8B txtBsSIdProp bsTxtIdPropprxyUtf8 verEncoding$fEncodeFTYPEEitherEncEnc$fEncodeFTYPEEitherEncEnc0 NonAsciiChar prxyAscii encodeImpl$fEncodeFTYPEEitherEncEnc1$fEncodeFTYPEEitherEncEnc2$fDecodeFTYPEfEncEnc3$fSubsetSymbol"r-ASCII""r-UTF8"$fEncodeFTYPEEitherEncEnc3$fEqNonAsciiChar$fShowNonAsciiChar eHelloAsciiB helloAsciiB helloAsciiT helloUtf8B helloUtf8B64B helloUtf8B64TnotTextBlenientSomething b64IsAscii encodeSign decodeSign helloSigned propEncDechacker prxyMySignConfig sizeLimithelloB64helloB64DecodedhelloB64Recovered helloB64B64helloB64B64PartDecodehelloB64B64DecodedhelloB64B64RecoveredErrhelloUPP helloTitleRev exampleConf helloTitle helloRevLimit helloLimitB64helloRevLimitParDec helloAscii helloAsciiB64helloAsciiB64PartDec$fHasAConfigSizeLimit $fShowConfig exAsciiTEexAsciiTmodifiedAsciiT toLowerAscii appendAscii text-1.2.3.1Data.Text.InternalTextbytestring-0.10.8.2Data.ByteString.Internal ByteString Data.TexttoLowerversion getBinDir getLibDir getDynLibDir getDataDir getLibexecDir getSysconfDirgetDataFileName